[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:27 dist-test-slave-0pn3 krb5kdc[12165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:27 dist-test-slave-0pn3 krb5kdc[12165](info): set up 2 sockets
Jun 21 07:56:27 dist-test-slave-0pn3 krb5kdc[12165](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260621 07:56:28.432986 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:28 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:28.542932 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44631
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:44631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:28.643808 12178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:28.644016 12178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:28.644063 12178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:28.647727 12178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:28.647770 12178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:28.647789 12178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:28.647804 12178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:28.647819 12178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:28.652575 12178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:44631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:28.653354 12178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:28.654157 12178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:28.661748 12183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:28.661708 12184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:28.661841 12178 server_base.cc:1034] running on GCE node
W20260621 07:56:28.661721 12186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:28.662564 12178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:28.663460 12178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:28.664623 12178 hybrid_clock.cc:648] HybridClock initialized: now 1782028588664602 us; error 34 us; skew 500 ppm
Jun 21 07:56:28 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:28.667116 12178 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:28.667941 12178 webserver.cc:460] Webserver started at http://127.11.223.190:37629/ using document root <none> and password file <none>
I20260621 07:56:28.668519 12178 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:28.668578 12178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:28.668807 12178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:28.670674 12178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "44a892bef7ba44ba8818f172f37e0f43"
format_stamp: "Formatted at 2026-06-21 07:56:28 on dist-test-slave-0pn3"
server_key: "05c0472c47085ed2d780d0aec0d38226"
server_key_iv: "f657bff9382ee974a8114aebdbf33f49"
server_key_version: "encryptionkey@0"
I20260621 07:56:28.671201 12178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "44a892bef7ba44ba8818f172f37e0f43"
format_stamp: "Formatted at 2026-06-21 07:56:28 on dist-test-slave-0pn3"
server_key: "05c0472c47085ed2d780d0aec0d38226"
server_key_iv: "f657bff9382ee974a8114aebdbf33f49"
server_key_version: "encryptionkey@0"
I20260621 07:56:28.674723 12178 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:28.676883 12193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:28.677973 12178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260621 07:56:28.678093 12178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "44a892bef7ba44ba8818f172f37e0f43"
format_stamp: "Formatted at 2026-06-21 07:56:28 on dist-test-slave-0pn3"
server_key: "05c0472c47085ed2d780d0aec0d38226"
server_key_iv: "f657bff9382ee974a8114aebdbf33f49"
server_key_version: "encryptionkey@0"
I20260621 07:56:28.678185 12178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:28.689805 12178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:28.692685 12178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:28.692862 12178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:28.699203 12178 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:44631
I20260621 07:56:28.699215 12245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:44631 every 8 connection(s)
I20260621 07:56:28.699977 12178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:28.702503 12246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:28.707453 12246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43: Bootstrap starting.
I20260621 07:56:28.708146 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12178
I20260621 07:56:28.708298 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:28.709852 12158 external_mini_cluster.cc:1428] Setting key 2fea6d066d2274f8fdaafa84eaf9a80c
I20260621 07:56:28.710510 12246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:28.711372 12246 log.cc:824] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:28.713284 12246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43: No bootstrap required, opened a new log
I20260621 07:56:28.717057 12246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a892bef7ba44ba8818f172f37e0f43" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44631 } }
I20260621 07:56:28.717293 12246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:28.717340 12246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a892bef7ba44ba8818f172f37e0f43, State: Initialized, Role: FOLLOWER
I20260621 07:56:28.717794 12246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a892bef7ba44ba8818f172f37e0f43" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44631 } }
I20260621 07:56:28.717957 12246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:28.718048 12246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:28.718147 12246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:28.718964 12246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a892bef7ba44ba8818f172f37e0f43" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44631 } }
I20260621 07:56:28.719255 12246 leader_election.cc:302] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44a892bef7ba44ba8818f172f37e0f43; no voters:
I20260621 07:56:28.719542 12246 leader_election.cc:288] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:28.719588 12251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:28.719743 12251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [term 1 LEADER]: Becoming Leader. State: Replica: 44a892bef7ba44ba8818f172f37e0f43, State: Running, Role: LEADER
I20260621 07:56:28.720036 12251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a892bef7ba44ba8818f172f37e0f43" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44631 } }
I20260621 07:56:28.720618 12246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:28.721613 12252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44a892bef7ba44ba8818f172f37e0f43. Latest consensus state: current_term: 1 leader_uuid: "44a892bef7ba44ba8818f172f37e0f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a892bef7ba44ba8818f172f37e0f43" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44631 } } }
I20260621 07:56:28.721727 12252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:28.722002 12253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44a892bef7ba44ba8818f172f37e0f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a892bef7ba44ba8818f172f37e0f43" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44631 } } }
I20260621 07:56:28.722090 12253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43 [sys.catalog]: This master's current role is: LEADER
W20260621 07:56:28.725423 12267 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:56:28.725494 12267 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:56:28.725562 12260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 21 07:56:28 dist-test-slave-0pn3 krb5kdc[12165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:28.726315 12260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:28.732427 12260 catalog_manager.cc:1347] Generated new cluster ID: d229c4cc0e53426ca17c77a28ac6ebe8
I20260621 07:56:28.732441 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:28.713015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.11.223.190:44631)
0621 07:56:28.713458 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:28.713477 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:28.714524 (+ 1047us) server_negotiation.cc:407] Connection header received
0621 07:56:28.717040 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:28.717056 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:28.717406 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:28.717742 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:28.719812 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:28.720986 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:28.722642 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:28.723344 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:28.723497 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:28.726869 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:28.726875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:28.726891 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:28.726913 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:28.728881 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:28.729518 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:28.729523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:28.729528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:28.729651 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:28.730014 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:28.730017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:28.730018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:28.730427 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:28.730617 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:28.731220 (+ 603us) server_negotiation.cc:299] Negotiation successful
0621 07:56:28.731527 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260621 07:56:28.732548 12260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:28.745042 12260 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:28.746066 12260 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:28.752511 12260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44a892bef7ba44ba8818f172f37e0f43: Generated new TSK 0
I20260621 07:56:28.753108 12260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:28.779551 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44631
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:28.893559 12272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:28.893759 12272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:28.893816 12272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:28.897615 12272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:28.897665 12272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:28.897738 12272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:28.902196 12272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44631
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:28.902947 12272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:28.903697 12272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:28.912022 12278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:28.912070 12280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:28.912114 12277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:28.912241 12272 server_base.cc:1034] running on GCE node
I20260621 07:56:28.912657 12272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:28.913204 12272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:28.914368 12272 hybrid_clock.cc:648] HybridClock initialized: now 1782028588914351 us; error 35 us; skew 500 ppm
Jun 21 07:56:28 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:28.916764 12272 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:28.917480 12272 webserver.cc:460] Webserver started at http://127.11.223.129:33035/ using document root <none> and password file <none>
I20260621 07:56:28.917999 12272 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:28.918083 12272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:28.918293 12272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:28.920115 12272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "ccee4b8cfb0f4ac5b091758beb334c31"
format_stamp: "Formatted at 2026-06-21 07:56:28 on dist-test-slave-0pn3"
server_key: "6ce2cd5ed08ab8b08d3b5df7cb4098e8"
server_key_iv: "f08ab64228d09ffcc11210a93ceb3287"
server_key_version: "encryptionkey@0"
I20260621 07:56:28.920658 12272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "ccee4b8cfb0f4ac5b091758beb334c31"
format_stamp: "Formatted at 2026-06-21 07:56:28 on dist-test-slave-0pn3"
server_key: "6ce2cd5ed08ab8b08d3b5df7cb4098e8"
server_key_iv: "f08ab64228d09ffcc11210a93ceb3287"
server_key_version: "encryptionkey@0"
I20260621 07:56:28.923949 12272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:28.926120 12287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:28.927103 12272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:28.927232 12272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "ccee4b8cfb0f4ac5b091758beb334c31"
format_stamp: "Formatted at 2026-06-21 07:56:28 on dist-test-slave-0pn3"
server_key: "6ce2cd5ed08ab8b08d3b5df7cb4098e8"
server_key_iv: "f08ab64228d09ffcc11210a93ceb3287"
server_key_version: "encryptionkey@0"
I20260621 07:56:28.927330 12272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:28.938381 12272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:28.940791 12272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:28.940968 12272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:28.941454 12272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:28.942194 12272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:28.942260 12272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:28.942322 12272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:28.942369 12272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:28.950244 12272 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:45203
I20260621 07:56:28.950268 12400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:45203 every 8 connection(s)
I20260621 07:56:28.951000 12272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:28.954905 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12272
I20260621 07:56:28.955006 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:28.955258 12158 external_mini_cluster.cc:1428] Setting key 46c8e774faa0929aa71177dde16ab2c2
Jun 21 07:56:28 dist-test-slave-0pn3 krb5kdc[12165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:28.964612 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:28.952572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:49939 (local address 127.11.223.190:44631)
0621 07:56:28.952689 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:28.952692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:28.953671 (+ 979us) server_negotiation.cc:407] Connection header received
0621 07:56:28.954392 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:28.954395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:28.954437 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:28.954508 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:28.955867 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:28.956524 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:28.958157 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:28.958417 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:28.958521 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:28.961187 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:28.961191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:28.961192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:28.961214 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:28.962576 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:28.963151 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:28.963154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:28.963155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:28.963224 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:28.963577 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:28.963579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:28.963580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:28.963729 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:28.963799 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:28.964410 (+ 611us) server_negotiation.cc:299] Negotiation successful
0621 07:56:28.964502 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:28.965595 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:28.952898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44631 (local address 127.11.223.129:49939)
0621 07:56:28.953535 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:28.953571 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:28.954226 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:28.954643 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:28.954649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:28.955131 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:28.955716 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:28.955731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:28.956636 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:28.956654 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:28.958040 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:28.958055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:28.958565 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:28.958569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:28.958721 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:28.959273 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:28.959287 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:28.961038 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:28.962683 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:28.962691 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:28.962706 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:28.963063 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:28.963305 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:28.963308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:28.963310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:28.963485 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:28.963824 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:28.963829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:28.964119 (+ 290us) client_negotiation.cc:769] Sending connection context
0621 07:56:28.964381 (+ 262us) client_negotiation.cc:240] Negotiation successful
0621 07:56:28.964709 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":113,"threads_started":1}
I20260621 07:56:28.966581 12401 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44631
I20260621 07:56:28.966795 12401 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:56:28.967270 12401 heartbeater.cc:511] Master 127.11.223.190:44631 requested a full tablet report, sending...
I20260621 07:56:28.968919 12210 ts_manager.cc:194] Registered new tserver with Master: ccee4b8cfb0f4ac5b091758beb334c31 (127.11.223.129:45203)
I20260621 07:56:28.969969 12210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:49939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:28.980746 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44631
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:29.079319 12406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:29.079514 12406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:29.079550 12406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:29.083258 12406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:29.083307 12406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:29.083415 12406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:29.088083 12406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44631
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:29.088953 12406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:29.089768 12406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:29.100741 12411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:29.100835 12406 server_base.cc:1034] running on GCE node
W20260621 07:56:29.100733 12414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:29.100733 12412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:29.101420 12406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:29.101943 12406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:29.103101 12406 hybrid_clock.cc:648] HybridClock initialized: now 1782028589103076 us; error 39 us; skew 500 ppm
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:29.105656 12406 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:29.106362 12406 webserver.cc:460] Webserver started at http://127.11.223.130:41817/ using document root <none> and password file <none>
I20260621 07:56:29.106835 12406 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:29.106916 12406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:29.107108 12406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:29.108983 12406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "0a073523f4b9484985d07d9576351055"
format_stamp: "Formatted at 2026-06-21 07:56:29 on dist-test-slave-0pn3"
server_key: "bfd49d53ebfdef90447974aec2dc5193"
server_key_iv: "533aa98ebd5cf8a60246b32bef4d4756"
server_key_version: "encryptionkey@0"
I20260621 07:56:29.109483 12406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "0a073523f4b9484985d07d9576351055"
format_stamp: "Formatted at 2026-06-21 07:56:29 on dist-test-slave-0pn3"
server_key: "bfd49d53ebfdef90447974aec2dc5193"
server_key_iv: "533aa98ebd5cf8a60246b32bef4d4756"
server_key_version: "encryptionkey@0"
I20260621 07:56:29.112653 12406 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:29.114593 12421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:29.115669 12406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:29.115792 12406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "0a073523f4b9484985d07d9576351055"
format_stamp: "Formatted at 2026-06-21 07:56:29 on dist-test-slave-0pn3"
server_key: "bfd49d53ebfdef90447974aec2dc5193"
server_key_iv: "533aa98ebd5cf8a60246b32bef4d4756"
server_key_version: "encryptionkey@0"
I20260621 07:56:29.115881 12406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:29.131570 12406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:29.133982 12406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:29.134202 12406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:29.134685 12406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:29.135406 12406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:29.135463 12406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:29.135519 12406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:29.135558 12406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:29.143457 12406 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:38541
I20260621 07:56:29.143491 12534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:38541 every 8 connection(s)
I20260621 07:56:29.144279 12406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:29.145179 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12406
I20260621 07:56:29.145304 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:29.145568 12158 external_mini_cluster.cc:1428] Setting key 95feb779c1d7c5ba6e535e84e8f67bb9
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 07:56:29.157505 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:29.146488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:56221 (local address 127.11.223.190:44631)
0621 07:56:29.146617 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:29.146622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:29.147604 (+ 982us) server_negotiation.cc:407] Connection header received
0621 07:56:29.148520 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:29.148526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:29.148588 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:29.148674 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:29.149740 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.150284 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.151424 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.151635 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.151737 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:29.153967 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:29.153975 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:29.153977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:29.153998 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:29.155211 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.156030 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.156032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.156034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.156110 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.156483 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.156488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.156489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.156658 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:29.156766 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:29.157220 (+ 454us) server_negotiation.cc:299] Negotiation successful
0621 07:56:29.157337 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:56:29.158178 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:29.146862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44631 (local address 127.11.223.130:56221)
0621 07:56:29.147475 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:29.147520 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:29.148289 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:29.148816 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:29.148822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:29.149141 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:29.149596 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:29.149607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.150409 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.150413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:29.151284 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:29.151293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.151767 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.151771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:29.151926 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:29.152484 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:29.152499 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:29.153813 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:29.155635 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:29.155639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:29.155649 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:29.155893 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:29.156206 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:29.156209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:29.156211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:29.156392 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:29.156764 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:29.156771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:29.156990 (+ 219us) client_negotiation.cc:769] Sending connection context
0621 07:56:29.157164 (+ 174us) client_negotiation.cc:240] Negotiation successful
0621 07:56:29.157437 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":117,"threads_started":1}
I20260621 07:56:29.159212 12535 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44631
I20260621 07:56:29.159472 12535 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:29.159987 12535 heartbeater.cc:511] Master 127.11.223.190:44631 requested a full tablet report, sending...
I20260621 07:56:29.161062 12210 ts_manager.cc:194] Registered new tserver with Master: 0a073523f4b9484985d07d9576351055 (127.11.223.130:38541)
I20260621 07:56:29.161543 12210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:56221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:29.169077 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44631
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:29.269666 12540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:29.269887 12540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:29.269946 12540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:29.273775 12540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:29.273844 12540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:29.273931 12540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:29.278638 12540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44631
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:29.279531 12540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:29.280434 12540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:29.289813 12546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:29.289819 12548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:29.289988 12540 server_base.cc:1034] running on GCE node
W20260621 07:56:29.289901 12545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:29.290454 12540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:29.291014 12540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:29.292193 12540 hybrid_clock.cc:648] HybridClock initialized: now 1782028589292170 us; error 45 us; skew 500 ppm
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:29.294667 12540 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:29.295367 12540 webserver.cc:460] Webserver started at http://127.11.223.131:45441/ using document root <none> and password file <none>
I20260621 07:56:29.295825 12540 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:29.295884 12540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:29.296028 12540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:29.297925 12540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "f99037c40bde46d3867a44ba9cafc095"
format_stamp: "Formatted at 2026-06-21 07:56:29 on dist-test-slave-0pn3"
server_key: "d147b45ef7cc43c99747bbb756fae8be"
server_key_iv: "0a3fba8d597c7849ce384c03b5199971"
server_key_version: "encryptionkey@0"
I20260621 07:56:29.298442 12540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "f99037c40bde46d3867a44ba9cafc095"
format_stamp: "Formatted at 2026-06-21 07:56:29 on dist-test-slave-0pn3"
server_key: "d147b45ef7cc43c99747bbb756fae8be"
server_key_iv: "0a3fba8d597c7849ce384c03b5199971"
server_key_version: "encryptionkey@0"
I20260621 07:56:29.301726 12540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260621 07:56:29.303876 12555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:29.305025 12540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:56:29.305132 12540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "f99037c40bde46d3867a44ba9cafc095"
format_stamp: "Formatted at 2026-06-21 07:56:29 on dist-test-slave-0pn3"
server_key: "d147b45ef7cc43c99747bbb756fae8be"
server_key_iv: "0a3fba8d597c7849ce384c03b5199971"
server_key_version: "encryptionkey@0"
I20260621 07:56:29.305223 12540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:29.314837 12540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:29.317411 12540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:29.317575 12540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:29.318086 12540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:29.318814 12540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:29.318897 12540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:29.318989 12540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:29.319033 12540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:29.327464 12540 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:44799
I20260621 07:56:29.327491 12668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:44799 every 8 connection(s)
I20260621 07:56:29.328214 12540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:29.333997 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12540
I20260621 07:56:29.334095 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:29.334338 12158 external_mini_cluster.cc:1428] Setting key fb6d9e74dde669e3bd6d919d7cd0c294
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:29.341043 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:29.329880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:43247 (local address 127.11.223.190:44631)
0621 07:56:29.329992 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:29.329996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:29.331138 (+ 1142us) server_negotiation.cc:407] Connection header received
0621 07:56:29.331975 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:29.331978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:29.332026 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:29.332100 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:29.333090 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.333571 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.335038 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.335330 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.335446 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:29.337736 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:29.337739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:29.337741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:29.337764 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:29.339124 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.339607 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.339612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.339613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.339690 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.340031 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.340036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.340038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.340208 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:29.340308 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:29.340761 (+ 453us) server_negotiation.cc:299] Negotiation successful
0621 07:56:29.340881 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 07:56:29.341682 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:29.330139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44631 (local address 127.11.223.131:43247)
0621 07:56:29.330995 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:29.331032 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:29.331787 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:29.332210 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:29.332216 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:29.332564 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:29.332974 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:29.332985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.333664 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.333666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:29.334906 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:29.334915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.335472 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.335475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:29.335596 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:29.336089 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:29.336106 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:29.337564 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:29.339243 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:29.339248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:29.339259 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:29.339509 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:29.339785 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:29.339788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:29.339790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:29.339917 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:29.340339 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:29.340346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:29.340548 (+ 202us) client_negotiation.cc:769] Sending connection context
0621 07:56:29.340718 (+ 170us) client_negotiation.cc:240] Negotiation successful
0621 07:56:29.340997 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":528,"threads_started":1}
I20260621 07:56:29.342605 12669 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44631
I20260621 07:56:29.342814 12669 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:29.343241 12669 heartbeater.cc:511] Master 127.11.223.190:44631 requested a full tablet report, sending...
I20260621 07:56:29.344105 12210 ts_manager.cc:194] Registered new tserver with Master: f99037c40bde46d3867a44ba9cafc095 (127.11.223.131:44799)
I20260621 07:56:29.344652 12210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:43247
I20260621 07:56:29.348368 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:29.370909 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:29.362811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.11.223.129:45203)
0621 07:56:29.363194 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:29.363203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:29.363223 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:56:29.363264 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:29.363268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:29.363408 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:29.363513 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:29.364066 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.364798 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.365491 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.365780 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.365860 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:29.368027 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:29.368034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:29.368043 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:29.368064 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:29.369436 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.369785 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.369788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.369789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.369877 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.370077 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.370079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.370080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.370287 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:29.370423 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:29.370570 (+ 147us) server_negotiation.cc:299] Negotiation successful
0621 07:56:29.370706 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":146,"threads_started":1}
W20260621 07:56:29.371701 12315 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51714
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:29.400580 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:29.390396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.11.223.129:45203)
0621 07:56:29.390508 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:29.390513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:29.390838 (+ 325us) server_negotiation.cc:407] Connection header received
0621 07:56:29.390999 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:29.391004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:29.391083 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:29.391303 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:29.392141 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.393120 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.394338 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:29.394699 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:29.394801 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:29.397321 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:29.397325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:29.397329 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:29.397352 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:29.398841 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.399366 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.399369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.399371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.399435 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:29.399717 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:29.399720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:29.399722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:29.399940 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:29.400047 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:29.400305 (+ 258us) server_negotiation.cc:299] Negotiation successful
0621 07:56:29.400429 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:56:29.401516 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12272
I20260621 07:56:29.410125 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12406
I20260621 07:56:29.418340 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12540
I20260621 07:56:29.423439 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12178
2026-06-21T07:56:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2009 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12683](info): set up 2 sockets
Jun 21 07:56:29 dist-test-slave-0pn3 krb5kdc[12683](info): commencing operation
krb5kdc: starting...
W20260621 07:56:30.412297 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:30 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:30.517168 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:37249
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:30.618029 12696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:30.618270 12696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:30.618345 12696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:30.622674 12696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:30.622740 12696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:30.622774 12696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:30.622838 12696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:30.622886 12696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:30.627754 12696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:37249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:37249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:30.628696 12696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:30.629568 12696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:30.637328 12704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:30.637358 12702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:30.637334 12701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:30.637427 12696 server_base.cc:1034] running on GCE node
I20260621 07:56:30.638077 12696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:30.638998 12696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:30.640158 12696 hybrid_clock.cc:648] HybridClock initialized: now 1782028590640133 us; error 37 us; skew 500 ppm
Jun 21 07:56:30 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028590, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:30.642805 12696 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:30.643579 12696 webserver.cc:460] Webserver started at http://127.11.223.190:45063/ using document root <none> and password file <none>
I20260621 07:56:30.644098 12696 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:30.644181 12696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:30.644441 12696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:30.646246 12696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "14e23b5a869a4f3bb39121141699641d"
format_stamp: "Formatted at 2026-06-21 07:56:30 on dist-test-slave-0pn3"
server_key: "85367c91d1ea609d083f1e5c23ec4476"
server_key_iv: "6e000a67ed8d9cb6054580682148a0a4"
server_key_version: "encryptionkey@0"
I20260621 07:56:30.646829 12696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "14e23b5a869a4f3bb39121141699641d"
format_stamp: "Formatted at 2026-06-21 07:56:30 on dist-test-slave-0pn3"
server_key: "85367c91d1ea609d083f1e5c23ec4476"
server_key_iv: "6e000a67ed8d9cb6054580682148a0a4"
server_key_version: "encryptionkey@0"
I20260621 07:56:30.650199 12696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:30.652433 12711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:30.653563 12696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:30.653687 12696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "14e23b5a869a4f3bb39121141699641d"
format_stamp: "Formatted at 2026-06-21 07:56:30 on dist-test-slave-0pn3"
server_key: "85367c91d1ea609d083f1e5c23ec4476"
server_key_iv: "6e000a67ed8d9cb6054580682148a0a4"
server_key_version: "encryptionkey@0"
I20260621 07:56:30.653787 12696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:30.675794 12696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:30.678328 12696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:30.678515 12696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:30.685179 12696 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:37249
I20260621 07:56:30.685191 12763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:37249 every 8 connection(s)
I20260621 07:56:30.686033 12696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:30.688627 12764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:30.691771 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12696
I20260621 07:56:30.691891 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:30.692121 12158 external_mini_cluster.cc:1428] Setting key af1c56bbfbc04ab72215347609c66e5c
I20260621 07:56:30.694063 12764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d: Bootstrap starting.
I20260621 07:56:30.696038 12764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:30.696753 12764 log.cc:824] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:30.698382 12764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d: No bootstrap required, opened a new log
Jun 21 07:56:30 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:30.702179 12764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e23b5a869a4f3bb39121141699641d" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 37249 } }
I20260621 07:56:30.702369 12764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:30.702422 12764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14e23b5a869a4f3bb39121141699641d, State: Initialized, Role: FOLLOWER
I20260621 07:56:30.702829 12764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e23b5a869a4f3bb39121141699641d" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 37249 } }
I20260621 07:56:30.702940 12764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:30.703013 12764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:30.703155 12764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:30.704047 12764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e23b5a869a4f3bb39121141699641d" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 37249 } }
I20260621 07:56:30.704406 12764 leader_election.cc:302] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14e23b5a869a4f3bb39121141699641d; no voters:
I20260621 07:56:30.704706 12764 leader_election.cc:288] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:30.704793 12769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:30.704996 12769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [term 1 LEADER]: Becoming Leader. State: Replica: 14e23b5a869a4f3bb39121141699641d, State: Running, Role: LEADER
I20260621 07:56:30.705356 12769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e23b5a869a4f3bb39121141699641d" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 37249 } }
I20260621 07:56:30.705637 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:30.693455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.11.223.190:37249)
0621 07:56:30.693881 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:30.693895 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:30.693928 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:56:30.694558 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:30.694576 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:30.694885 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:30.695198 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:30.695939 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:30.697022 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:30.697851 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:30.698319 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:30.698485 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:30.700980 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:30.700984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:30.700996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:30.701019 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:30.702659 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:30.703096 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:30.703106 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:30.703110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:30.703218 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:30.703550 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:30.703552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:30.703553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:30.703881 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:30.704013 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:30.704386 (+ 373us) server_negotiation.cc:299] Negotiation successful
0621 07:56:30.704692 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":123,"threads_started":1}
I20260621 07:56:30.705775 12764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:30.706427 12769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e23b5a869a4f3bb39121141699641d. Latest consensus state: current_term: 1 leader_uuid: "14e23b5a869a4f3bb39121141699641d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e23b5a869a4f3bb39121141699641d" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 37249 } } }
I20260621 07:56:30.706600 12769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:30.706840 12770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14e23b5a869a4f3bb39121141699641d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e23b5a869a4f3bb39121141699641d" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 37249 } } }
I20260621 07:56:30.706944 12770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:30.707641 12778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:30.710227 12778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:30.714968 12778 catalog_manager.cc:1347] Generated new cluster ID: d25a6b2bbbd042209ff08bb2efde61aa
I20260621 07:56:30.715027 12778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:30.747450 12778 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:30.748414 12778 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:30.755354 12778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14e23b5a869a4f3bb39121141699641d: Generated new TSK 0
I20260621 07:56:30.756091 12778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:30.789425 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:37249
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:30.891072 12790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:30.891279 12790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:30.891319 12790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:30.895277 12790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:30.895324 12790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:30.895388 12790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:30.900086 12790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:37249
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:30.900923 12790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:30.901746 12790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:30.910439 12796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:30.910482 12798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:30.910660 12790 server_base.cc:1034] running on GCE node
W20260621 07:56:30.910439 12795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:30.911034 12790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:30.911610 12790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:30.912777 12790 hybrid_clock.cc:648] HybridClock initialized: now 1782028590912762 us; error 27 us; skew 500 ppm
Jun 21 07:56:30 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028590, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:30.915352 12790 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:30.916081 12790 webserver.cc:460] Webserver started at http://127.11.223.129:39533/ using document root <none> and password file <none>
I20260621 07:56:30.916621 12790 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:30.916702 12790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:30.916899 12790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:30.918699 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "81d19f58498541bba809109ec78a0c04"
format_stamp: "Formatted at 2026-06-21 07:56:30 on dist-test-slave-0pn3"
server_key: "5c93a38e480f0d24dd94331554b771c4"
server_key_iv: "c3c67e224f8596efd7e600dcd0129aae"
server_key_version: "encryptionkey@0"
I20260621 07:56:30.919225 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "81d19f58498541bba809109ec78a0c04"
format_stamp: "Formatted at 2026-06-21 07:56:30 on dist-test-slave-0pn3"
server_key: "5c93a38e480f0d24dd94331554b771c4"
server_key_iv: "c3c67e224f8596efd7e600dcd0129aae"
server_key_version: "encryptionkey@0"
I20260621 07:56:30.922457 12790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260621 07:56:30.924537 12805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:30.925523 12790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 07:56:30.925634 12790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "81d19f58498541bba809109ec78a0c04"
format_stamp: "Formatted at 2026-06-21 07:56:30 on dist-test-slave-0pn3"
server_key: "5c93a38e480f0d24dd94331554b771c4"
server_key_iv: "c3c67e224f8596efd7e600dcd0129aae"
server_key_version: "encryptionkey@0"
I20260621 07:56:30.925724 12790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:30.943661 12790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:30.946620 12790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:30.946810 12790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:30.947336 12790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:30.948074 12790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:30.948143 12790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:30.948215 12790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:30.948259 12790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:30.955981 12790 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:37299
I20260621 07:56:30.956034 12918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:37299 every 8 connection(s)
I20260621 07:56:30.956869 12790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:30.964254 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12790
I20260621 07:56:30.964394 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:30.964632 12158 external_mini_cluster.cc:1428] Setting key 76b989a46225270ef7be193f7e9d5bee
Jun 21 07:56:30 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028590, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:30.969594 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:30.958561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52139 (local address 127.11.223.190:37249)
0621 07:56:30.958690 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:30.958694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:30.959431 (+ 737us) server_negotiation.cc:407] Connection header received
0621 07:56:30.960618 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:30.960621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:30.960664 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:30.960734 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:30.961823 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:30.962325 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:30.963309 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:30.963514 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:30.963587 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:30.966033 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:30.966037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:30.966039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:30.966064 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:30.967482 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:30.967977 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:30.967983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:30.967988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:30.968087 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:30.968454 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:30.968460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:30.968464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:30.968701 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:30.968804 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:30.969230 (+ 426us) server_negotiation.cc:299] Negotiation successful
0621 07:56:30.969366 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:56:30.970158 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:30.958894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:37249 (local address 127.11.223.129:52139)
0621 07:56:30.959293 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:30.959328 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:30.960406 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:30.960968 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:30.960973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:30.961286 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:30.961680 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:30.961691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:30.962432 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:30.962434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:30.963199 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:30.963206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:30.963624 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:30.963626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:30.963794 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:30.964361 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:30.964380 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:30.965893 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:30.967597 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:30.967602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:30.967612 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:30.967883 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:30.968182 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:30.968184 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:30.968186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:30.968376 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:30.968794 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:30.968798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:30.968989 (+ 191us) client_negotiation.cc:769] Sending connection context
0621 07:56:30.969144 (+ 155us) client_negotiation.cc:240] Negotiation successful
0621 07:56:30.969366 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":110,"threads_started":1}
I20260621 07:56:30.971330 12919 heartbeater.cc:347] Connected to a master server at 127.11.223.190:37249
I20260621 07:56:30.971573 12919 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:30.972050 12919 heartbeater.cc:511] Master 127.11.223.190:37249 requested a full tablet report, sending...
I20260621 07:56:30.973477 12728 ts_manager.cc:194] Registered new tserver with Master: 81d19f58498541bba809109ec78a0c04 (127.11.223.129:37299)
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:30.974706 12728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:52139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:30.989023 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:37249
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:31.090241 12924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:31.090456 12924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:31.090524 12924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:31.096675 12924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:31.096735 12924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:31.096836 12924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:31.101687 12924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:37249
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:31.102501 12924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:31.103338 12924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:31.112253 12932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:31.112250 12929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:31.112253 12930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:31.112534 12924 server_base.cc:1034] running on GCE node
I20260621 07:56:31.112948 12924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:31.113531 12924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:31.114698 12924 hybrid_clock.cc:648] HybridClock initialized: now 1782028591114688 us; error 30 us; skew 500 ppm
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:31.117159 12924 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:31.117934 12924 webserver.cc:460] Webserver started at http://127.11.223.130:39753/ using document root <none> and password file <none>
I20260621 07:56:31.118443 12924 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:31.118525 12924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:31.118728 12924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:31.120641 12924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "31724b5ab9bf47a0aec067d35ffab962"
format_stamp: "Formatted at 2026-06-21 07:56:31 on dist-test-slave-0pn3"
server_key: "89e71460b1021d1924cdd68246ab4b75"
server_key_iv: "c975d297b2b6bd457f632b03814f2682"
server_key_version: "encryptionkey@0"
I20260621 07:56:31.121158 12924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "31724b5ab9bf47a0aec067d35ffab962"
format_stamp: "Formatted at 2026-06-21 07:56:31 on dist-test-slave-0pn3"
server_key: "89e71460b1021d1924cdd68246ab4b75"
server_key_iv: "c975d297b2b6bd457f632b03814f2682"
server_key_version: "encryptionkey@0"
I20260621 07:56:31.124385 12924 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260621 07:56:31.126520 12939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:31.127607 12924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 07:56:31.127733 12924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "31724b5ab9bf47a0aec067d35ffab962"
format_stamp: "Formatted at 2026-06-21 07:56:31 on dist-test-slave-0pn3"
server_key: "89e71460b1021d1924cdd68246ab4b75"
server_key_iv: "c975d297b2b6bd457f632b03814f2682"
server_key_version: "encryptionkey@0"
I20260621 07:56:31.127828 12924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:31.142501 12924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:31.145069 12924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:31.145195 12924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:31.145673 12924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:31.146365 12924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:31.146410 12924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:31.146459 12924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:31.146476 12924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:31.154306 12924 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:35635
I20260621 07:56:31.154356 13052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:35635 every 8 connection(s)
I20260621 07:56:31.155098 12924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:31.164602 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 12924
I20260621 07:56:31.164719 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:31.165022 12158 external_mini_cluster.cc:1428] Setting key a3cd3e4a9b2837330ee7fca86c81615f
I20260621 07:56:31.169087 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.156732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:35597 (local address 127.11.223.190:37249)
0621 07:56:31.156873 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.156876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.157657 (+ 781us) server_negotiation.cc:407] Connection header received
0621 07:56:31.158916 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.158918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.158967 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.159033 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:31.160062 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.160595 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.161690 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.161911 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.162000 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.164138 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:31.164141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:31.164143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:31.164163 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:31.165701 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.166193 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.166199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.166203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.166289 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.168022 (+ 1733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.168028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.168030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.168269 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:31.168421 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.168842 (+ 421us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.168959 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:56:31.170099 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.157116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:37249 (local address 127.11.223.130:35597)
0621 07:56:31.157531 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.157566 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.158718 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.159199 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.159205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.159526 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:31.159954 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.159966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.160706 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.160709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.161554 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.161562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.162039 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.162041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.162167 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.162693 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:31.162707 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:31.164004 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:31.165787 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:31.165792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:31.165804 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:31.166097 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:31.167777 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:31.167779 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:31.167780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:31.167927 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:31.168419 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:31.168423 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:31.168622 (+ 199us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.168769 (+ 147us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.169116 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":124,"threads_started":1}
I20260621 07:56:31.171161 13053 heartbeater.cc:347] Connected to a master server at 127.11.223.190:37249
I20260621 07:56:31.171384 13053 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:31.171831 13053 heartbeater.cc:511] Master 127.11.223.190:37249 requested a full tablet report, sending...
I20260621 07:56:31.172842 12728 ts_manager.cc:194] Registered new tserver with Master: 31724b5ab9bf47a0aec067d35ffab962 (127.11.223.130:35635)
I20260621 07:56:31.173373 12728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:35597
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:31.188585 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:37249
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:31.290988 13058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:31.291249 13058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:31.291319 13058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:31.295189 13058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:31.295277 13058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:31.295363 13058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:31.300345 13058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:37249
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:31.301296 13058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:31.302194 13058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:31.311568 13064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:31.311582 13063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:31.311605 13066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:31.311628 13058 server_base.cc:1034] running on GCE node
I20260621 07:56:31.312238 13058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:31.312866 13058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:31.314045 13058 hybrid_clock.cc:648] HybridClock initialized: now 1782028591314028 us; error 28 us; skew 500 ppm
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:31.316602 13058 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:31.317365 13058 webserver.cc:460] Webserver started at http://127.11.223.131:34719/ using document root <none> and password file <none>
I20260621 07:56:31.317925 13058 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:31.318008 13058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:31.318217 13058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:31.320073 13058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "28f04636dc1c4628a916c0527f484de6"
format_stamp: "Formatted at 2026-06-21 07:56:31 on dist-test-slave-0pn3"
server_key: "61be21ac380a186dda23a9f242b13b57"
server_key_iv: "1b10fa43e502c7b7ade964396cff23b1"
server_key_version: "encryptionkey@0"
I20260621 07:56:31.320606 13058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "28f04636dc1c4628a916c0527f484de6"
format_stamp: "Formatted at 2026-06-21 07:56:31 on dist-test-slave-0pn3"
server_key: "61be21ac380a186dda23a9f242b13b57"
server_key_iv: "1b10fa43e502c7b7ade964396cff23b1"
server_key_version: "encryptionkey@0"
I20260621 07:56:31.323927 13058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:56:31.326134 13073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:31.327270 13058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:31.327392 13058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "28f04636dc1c4628a916c0527f484de6"
format_stamp: "Formatted at 2026-06-21 07:56:31 on dist-test-slave-0pn3"
server_key: "61be21ac380a186dda23a9f242b13b57"
server_key_iv: "1b10fa43e502c7b7ade964396cff23b1"
server_key_version: "encryptionkey@0"
I20260621 07:56:31.327492 13058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:31.342756 13058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:31.345382 13058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:31.345558 13058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:31.346060 13058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:31.346824 13058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:31.346887 13058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:31.346951 13058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:31.346997 13058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:31.354918 13058 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:37101
I20260621 07:56:31.354954 13186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:37101 every 8 connection(s)
I20260621 07:56:31.355698 13058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:31.363833 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13058
I20260621 07:56:31.363947 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:31.364197 12158 external_mini_cluster.cc:1428] Setting key 4b940b8612203247f00983d8689b117d
I20260621 07:56:31.367491 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.357193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:36749 (local address 127.11.223.190:37249)
0621 07:56:31.357330 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.357335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.358072 (+ 737us) server_negotiation.cc:407] Connection header received
0621 07:56:31.359325 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.359327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.359370 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.359439 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:31.360398 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.360896 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.361886 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.362068 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.362153 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.364426 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:31.364429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:31.364431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:31.364452 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:31.365884 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.366401 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.366403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.366404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.366477 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.366746 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.366748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.366749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.366891 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:31.366957 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.367310 (+ 353us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.367391 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:56:31.368175 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.357542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:37249 (local address 127.11.223.131:36749)
0621 07:56:31.357945 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.357983 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.359147 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.359571 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.359576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.359875 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:31.360265 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.360276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.360990 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.360992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.361773 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.361781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.362192 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.362194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.362312 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.362824 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:31.362838 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:31.364192 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:31.366010 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:31.366015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:31.366027 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:31.366319 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:31.366560 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:31.366562 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:31.366564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:31.366671 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:31.366963 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:31.366967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:31.367141 (+ 174us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.367270 (+ 129us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.367479 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":111,"threads_started":1}
I20260621 07:56:31.369184 13187 heartbeater.cc:347] Connected to a master server at 127.11.223.190:37249
I20260621 07:56:31.369345 13187 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:31.369717 13187 heartbeater.cc:511] Master 127.11.223.190:37249 requested a full tablet report, sending...
I20260621 07:56:31.370564 12728 ts_manager.cc:194] Registered new tserver with Master: 28f04636dc1c4628a916c0527f484de6 (127.11.223.131:37101)
I20260621 07:56:31.371052 12728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:36749
I20260621 07:56:31.377568 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:56:31.386749 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.380688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.11.223.190:37249)
0621 07:56:31.380792 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.380795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.380946 (+ 151us) server_negotiation.cc:407] Connection header received
0621 07:56:31.381032 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.381034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.381066 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.381129 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:31.381693 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.382184 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.382903 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.383094 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.383169 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.384310 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:31.384345 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:31.384351 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:31.384374 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:31.385570 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.385915 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.385917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.385918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.385979 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.386168 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.386170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.386171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.386334 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:31.386425 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.386578 (+ 153us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.386653 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:56:31.390609 12728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:31.393404 12728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:31.406476 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.401449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.11.223.131:37101)
0621 07:56:31.401773 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.401787 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.401808 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:56:31.401879 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.401886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.402450 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.402598 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:31.402775 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.403525 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.405176 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.405918 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.406065 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.406175 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.406245 (+ 70us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.406331 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":134,"threads_started":1}
I20260621 07:56:31.406466 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.400881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:37101 (local address 127.0.0.1:60984)
0621 07:56:31.401486 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.401500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.401583 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.402583 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.402586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.402603 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:31.402685 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.402690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.403639 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.403645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.405067 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.405075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.406035 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.406039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.406126 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.406147 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.406219 (+ 72us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.406339 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":229,"thread_start_us":77,"threads_started":1}
I20260621 07:56:31.406682 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.400886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.11.223.129:37299)
0621 07:56:31.401642 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.401653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.401674 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:56:31.401748 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.401753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.401940 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.402095 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:31.402494 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.403579 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.404963 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.405690 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.405816 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.405940 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.406308 (+ 368us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.406407 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":105,"threads_started":1}
I20260621 07:56:31.406776 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.401926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.11.223.130:35635)
0621 07:56:31.402297 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.402309 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.402329 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:56:31.402398 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.402403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.402594 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.402732 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:31.403002 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.403980 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.405327 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.405918 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.406034 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.406147 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.406527 (+ 380us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.406632 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":90,"threads_started":1}
I20260621 07:56:31.407662 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.401399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35635 (local address 127.0.0.1:39000)
0621 07:56:31.402021 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.402033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.402105 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.402817 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.402819 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.402835 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:31.402897 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.402902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.404109 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.404112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.405197 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.405205 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.406204 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.406208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.406289 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.406319 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.406382 (+ 63us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.406440 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":13,"thread_start_us":76,"threads_started":1}
I20260621 07:56:31.407991 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.400743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:37299 (local address 127.0.0.1:41600)
0621 07:56:31.401194 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.401228 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.401352 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.402226 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.402232 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.402251 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:31.402374 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.402381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.403721 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.403724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.404817 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.404825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.405914 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.405918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.406066 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.406104 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.406202 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.407847 (+ 1645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":1597,"thread_start_us":74,"threads_started":1}
I20260621 07:56:31.409720 13121 tablet_service.cc:1467] Processing CreateTablet for tablet e42643ff75d24c52b06ef28d946e8ed2 (DEFAULT_TABLE table=test-table [id=b275d04a9ba34aa486fce9d4c5438e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:31.410249 12987 tablet_service.cc:1467] Processing CreateTablet for tablet e42643ff75d24c52b06ef28d946e8ed2 (DEFAULT_TABLE table=test-table [id=b275d04a9ba34aa486fce9d4c5438e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:31.410800 13121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42643ff75d24c52b06ef28d946e8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:31.410557 12848 tablet_service.cc:1467] Processing CreateTablet for tablet e42643ff75d24c52b06ef28d946e8ed2 (DEFAULT_TABLE table=test-table [id=b275d04a9ba34aa486fce9d4c5438e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:31.411151 12987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42643ff75d24c52b06ef28d946e8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:31.411410 12848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42643ff75d24c52b06ef28d946e8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:31.416793 13204 tablet_bootstrap.cc:492] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962: Bootstrap starting.
I20260621 07:56:31.418291 13206 tablet_bootstrap.cc:492] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: Bootstrap starting.
I20260621 07:56:31.418368 13205 tablet_bootstrap.cc:492] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6: Bootstrap starting.
I20260621 07:56:31.419516 13204 tablet_bootstrap.cc:654] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:31.420193 13204 log.cc:824] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:31.420274 13205 tablet_bootstrap.cc:654] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:31.420718 13206 tablet_bootstrap.cc:654] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:31.421502 13206 log.cc:824] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:31.421502 13205 log.cc:824] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:31.421974 13204 tablet_bootstrap.cc:492] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962: No bootstrap required, opened a new log
I20260621 07:56:31.422173 13204 ts_tablet_manager.cc:1397] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 07:56:31.423281 13205 tablet_bootstrap.cc:492] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6: No bootstrap required, opened a new log
I20260621 07:56:31.423447 13205 ts_tablet_manager.cc:1397] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260621 07:56:31.423708 13206 tablet_bootstrap.cc:492] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: No bootstrap required, opened a new log
I20260621 07:56:31.423892 13206 ts_tablet_manager.cc:1397] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 07:56:31.425355 13204 raft_consensus.cc:348] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.425571 13204 raft_consensus.cc:374] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:31.425619 13204 raft_consensus.cc:729] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31724b5ab9bf47a0aec067d35ffab962, State: Initialized, Role: FOLLOWER
I20260621 07:56:31.426044 13204 consensus_queue.cc:260] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.426623 13053 heartbeater.cc:503] Master 127.11.223.190:37249 was elected leader, sending a full tablet report...
I20260621 07:56:31.426949 13204 ts_tablet_manager.cc:1428] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 07:56:31.426952 13205 raft_consensus.cc:348] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.427194 13205 raft_consensus.cc:374] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:31.427249 13205 raft_consensus.cc:729] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f04636dc1c4628a916c0527f484de6, State: Initialized, Role: FOLLOWER
I20260621 07:56:31.427356 13206 raft_consensus.cc:348] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.427561 13206 raft_consensus.cc:374] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:31.427609 13206 raft_consensus.cc:729] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d19f58498541bba809109ec78a0c04, State: Initialized, Role: FOLLOWER
I20260621 07:56:31.427630 13205 consensus_queue.cc:260] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.428239 13187 heartbeater.cc:503] Master 127.11.223.190:37249 was elected leader, sending a full tablet report...
I20260621 07:56:31.428527 13205 ts_tablet_manager.cc:1428] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260621 07:56:31.428031 13206 consensus_queue.cc:260] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.429119 12919 heartbeater.cc:503] Master 127.11.223.190:37249 was elected leader, sending a full tablet report...
I20260621 07:56:31.430490 13206 ts_tablet_manager.cc:1428] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
W20260621 07:56:31.457924 12920 tablet.cc:2369] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:31.474406 13212 raft_consensus.cc:482] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:31.474552 13212 raft_consensus.cc:504] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.475497 13212 leader_election.cc:288] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28f04636dc1c4628a916c0527f484de6 (127.11.223.131:37101), 31724b5ab9bf47a0aec067d35ffab962 (127.11.223.130:35635)
I20260621 07:56:31.480043 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.475821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:56859 (local address 127.11.223.130:35635)
0621 07:56:31.475939 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.475943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.476078 (+ 135us) server_negotiation.cc:407] Connection header received
0621 07:56:31.476353 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.476356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.476411 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.476504 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:31.476900 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.477505 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.479170 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.479570 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.479661 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.479699 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.479867 (+ 168us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.479937 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 07:56:31.480064 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.475690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35635 (local address 127.11.223.129:56859)
0621 07:56:31.475969 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.475983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.476125 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.476530 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.476535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.476559 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:31.476795 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.476802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.477604 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.477610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.479068 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.479079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.479669 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.479671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.479739 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.479758 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.479849 (+ 91us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.479915 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":117,"spinlock_wait_cycles":51712,"thread_start_us":58,"threads_started":1}
I20260621 07:56:31.480721 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.475690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:37101 (local address 127.11.223.129:55663)
0621 07:56:31.476085 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:31.476099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:31.476212 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:31.476577 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:31.476579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:31.476633 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:31.476831 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.476837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.477648 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.477651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.479069 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:31.479079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.480363 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.480370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:31.480467 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.480496 (+ 29us) client_negotiation.cc:769] Sending connection context
0621 07:56:31.480575 (+ 79us) client_negotiation.cc:240] Negotiation successful
0621 07:56:31.480624 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":165,"spinlock_wait_cycles":1792,"thread_start_us":112,"threads_started":1}
I20260621 07:56:31.480777 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.475776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:55663 (local address 127.11.223.131:37101)
0621 07:56:31.475902 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.475906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.476215 (+ 309us) server_negotiation.cc:407] Connection header received
0621 07:56:31.476385 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.476388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.476449 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.476566 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:31.476957 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.477533 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.479744 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.480201 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.480377 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.480423 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.480598 (+ 175us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.480666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:56:31.480805 13007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42643ff75d24c52b06ef28d946e8ed2" candidate_uuid: "81d19f58498541bba809109ec78a0c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31724b5ab9bf47a0aec067d35ffab962" is_pre_election: true
I20260621 07:56:31.481051 13007 raft_consensus.cc:2454] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d19f58498541bba809109ec78a0c04 in term 0.
I20260621 07:56:31.481335 13141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42643ff75d24c52b06ef28d946e8ed2" candidate_uuid: "81d19f58498541bba809109ec78a0c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28f04636dc1c4628a916c0527f484de6" is_pre_election: true
I20260621 07:56:31.481521 12806 leader_election.cc:302] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31724b5ab9bf47a0aec067d35ffab962, 81d19f58498541bba809109ec78a0c04; no voters:
I20260621 07:56:31.481621 13141 raft_consensus.cc:2454] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d19f58498541bba809109ec78a0c04 in term 0.
I20260621 07:56:31.481765 13212 raft_consensus.cc:2781] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:31.481863 13212 raft_consensus.cc:482] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:31.481930 13212 raft_consensus.cc:3037] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:31.482918 13212 raft_consensus.cc:504] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.483255 13212 leader_election.cc:288] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [CANDIDATE]: Term 1 election: Requested vote from peers 28f04636dc1c4628a916c0527f484de6 (127.11.223.131:37101), 31724b5ab9bf47a0aec067d35ffab962 (127.11.223.130:35635)
I20260621 07:56:31.483512 13141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42643ff75d24c52b06ef28d946e8ed2" candidate_uuid: "81d19f58498541bba809109ec78a0c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28f04636dc1c4628a916c0527f484de6"
I20260621 07:56:31.483579 13007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42643ff75d24c52b06ef28d946e8ed2" candidate_uuid: "81d19f58498541bba809109ec78a0c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31724b5ab9bf47a0aec067d35ffab962"
I20260621 07:56:31.483630 13141 raft_consensus.cc:3037] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:31.483701 13007 raft_consensus.cc:3037] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:31.484674 13141 raft_consensus.cc:2454] T e42643ff75d24c52b06ef28d946e8ed2 P 28f04636dc1c4628a916c0527f484de6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d19f58498541bba809109ec78a0c04 in term 1.
I20260621 07:56:31.484715 13007 raft_consensus.cc:2454] T e42643ff75d24c52b06ef28d946e8ed2 P 31724b5ab9bf47a0aec067d35ffab962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d19f58498541bba809109ec78a0c04 in term 1.
I20260621 07:56:31.484969 12809 leader_election.cc:302] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f04636dc1c4628a916c0527f484de6, 81d19f58498541bba809109ec78a0c04; no voters:
I20260621 07:56:31.485143 13212 raft_consensus.cc:2781] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:31.485351 13212 raft_consensus.cc:686] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [term 1 LEADER]: Becoming Leader. State: Replica: 81d19f58498541bba809109ec78a0c04, State: Running, Role: LEADER
I20260621 07:56:31.485636 13212 consensus_queue.cc:237] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } }
I20260621 07:56:31.488595 12727 catalog_manager.cc:5495] T e42643ff75d24c52b06ef28d946e8ed2 P 81d19f58498541bba809109ec78a0c04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81d19f58498541bba809109ec78a0c04 (127.11.223.129). New cstate: current_term: 1 leader_uuid: "81d19f58498541bba809109ec78a0c04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d19f58498541bba809109ec78a0c04" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 37299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28f04636dc1c4628a916c0527f484de6" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31724b5ab9bf47a0aec067d35ffab962" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35635 } health_report { overall_health: UNKNOWN } } }
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:31.505949 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.498098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.11.223.129:37299)
0621 07:56:31.498223 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.498227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.498391 (+ 164us) server_negotiation.cc:407] Connection header received
0621 07:56:31.498481 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.498483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.498516 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.498581 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:31.499195 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.499634 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.500419 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.500641 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.500734 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.502932 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:31.502940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:31.502951 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:31.502971 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:31.504531 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.504877 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.504881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.504882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.504961 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.505161 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.505163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.505164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.505405 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:31.505543 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.505674 (+ 131us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.505781 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:31.529968 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.522467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.11.223.129:37299)
0621 07:56:31.522573 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.522576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.522724 (+ 148us) server_negotiation.cc:407] Connection header received
0621 07:56:31.522830 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.522832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.522865 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.522922 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:31.523465 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.523904 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.524675 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.524867 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.524937 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.527154 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:31.527157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:31.527159 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:31.527178 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:31.528485 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.528850 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.528853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.528854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.528920 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.529178 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.529183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.529185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.529344 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:31.529495 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.529718 (+ 223us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.529823 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[12683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:31.552788 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:31.545921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.11.223.129:37299)
0621 07:56:31.546065 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:31.546068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:31.546133 (+ 65us) server_negotiation.cc:407] Connection header received
0621 07:56:31.546223 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:31.546225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:31.546258 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:31.546329 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:31.546866 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.547282 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.548017 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:31.548210 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:31.548280 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:31.550413 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:31.550416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:31.550418 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:31.550438 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:31.551563 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.551889 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.551892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.551893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.551955 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:31.552173 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:31.552175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:31.552176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:31.552305 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:31.552472 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:31.552608 (+ 136us) server_negotiation.cc:299] Negotiation successful
0621 07:56:31.552687 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260621 07:56:31.553990 12833 tablet_service.cc:2908] Rejecting scan request for tablet e42643ff75d24c52b06ef28d946e8ed2: Uninitialized: clean time has not yet been initialized
I20260621 07:56:31.555696 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12790
I20260621 07:56:31.561347 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12924
I20260621 07:56:31.566378 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13058
I20260621 07:56:31.571712 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 12696
2026-06-21T07:56:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2148 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[13229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[13229](info): set up 2 sockets
Jun 21 07:56:31 dist-test-slave-0pn3 krb5kdc[13229](info): commencing operation
krb5kdc: starting...
W20260621 07:56:32.572542 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:32 dist-test-slave-0pn3 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:32.679071 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43689
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:32.780108 13242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:32.780346 13242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:32.780424 13242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:32.784256 13242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:32.784298 13242 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260621 07:56:32.784369 13242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:32.784405 13242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:32.784422 13242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:32.784436 13242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:32.789163 13242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43689
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:32.789933 13242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:32.790740 13242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:32.798485 13247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:32.798578 13248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:32.798538 13250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:32.798528 13242 server_base.cc:1034] running on GCE node
I20260621 07:56:32.799216 13242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:32.800132 13242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:32.801301 13242 hybrid_clock.cc:648] HybridClock initialized: now 1782028592801279 us; error 45 us; skew 500 ppm
Jun 21 07:56:32 dist-test-slave-0pn3 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028592, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:32.806494 13242 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:32.807258 13242 webserver.cc:460] Webserver started at http://127.11.223.190:42123/ using document root <none> and password file <none>
I20260621 07:56:32.807776 13242 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:32.807859 13242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:32.808064 13242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:32.809916 13242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "db651c9b4cf6444ebfe4cd428e381f19"
format_stamp: "Formatted at 2026-06-21 07:56:32 on dist-test-slave-0pn3"
server_key: "75926c2ab3fcd3607935bd5c2e930101"
server_key_iv: "fae8bd7a88797d184a9706e6d3783835"
server_key_version: "encryptionkey@0"
I20260621 07:56:32.810457 13242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "db651c9b4cf6444ebfe4cd428e381f19"
format_stamp: "Formatted at 2026-06-21 07:56:32 on dist-test-slave-0pn3"
server_key: "75926c2ab3fcd3607935bd5c2e930101"
server_key_iv: "fae8bd7a88797d184a9706e6d3783835"
server_key_version: "encryptionkey@0"
I20260621 07:56:32.813791 13242 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:32.815811 13257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:32.816942 13242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:32.817070 13242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "db651c9b4cf6444ebfe4cd428e381f19"
format_stamp: "Formatted at 2026-06-21 07:56:32 on dist-test-slave-0pn3"
server_key: "75926c2ab3fcd3607935bd5c2e930101"
server_key_iv: "fae8bd7a88797d184a9706e6d3783835"
server_key_version: "encryptionkey@0"
I20260621 07:56:32.817166 13242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:32.835175 13242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:32.837633 13242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:32.837818 13242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:32.844918 13242 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:43689
I20260621 07:56:32.844928 13319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:43689 every 8 connection(s)
I20260621 07:56:32.845788 13242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:32.848217 13320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:32.853184 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: Bootstrap starting.
I20260621 07:56:32.853487 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13242
I20260621 07:56:32.853600 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:32.853822 12158 external_mini_cluster.cc:1428] Setting key 5fb8460099d6f94a531f977604b92b2b
I20260621 07:56:32.855470 13320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:32.856099 13320 log.cc:824] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:32.857740 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: No bootstrap required, opened a new log
I20260621 07:56:32.861035 13320 raft_consensus.cc:348] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db651c9b4cf6444ebfe4cd428e381f19" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43689 } }
I20260621 07:56:32.861320 13320 raft_consensus.cc:374] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:32.861383 13320 raft_consensus.cc:729] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db651c9b4cf6444ebfe4cd428e381f19, State: Initialized, Role: FOLLOWER
Jun 21 07:56:32 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:32.861819 13320 consensus_queue.cc:260] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db651c9b4cf6444ebfe4cd428e381f19" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43689 } }
I20260621 07:56:32.861972 13320 raft_consensus.cc:388] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:32.862041 13320 raft_consensus.cc:482] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:32.862138 13320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:32.863193 13320 raft_consensus.cc:504] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db651c9b4cf6444ebfe4cd428e381f19" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43689 } }
I20260621 07:56:32.863569 13320 leader_election.cc:302] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db651c9b4cf6444ebfe4cd428e381f19; no voters:
I20260621 07:56:32.863878 13320 leader_election.cc:288] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:32.863977 13325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:32.864180 13325 raft_consensus.cc:686] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [term 1 LEADER]: Becoming Leader. State: Replica: db651c9b4cf6444ebfe4cd428e381f19, State: Running, Role: LEADER
I20260621 07:56:32.864560 13325 consensus_queue.cc:237] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db651c9b4cf6444ebfe4cd428e381f19" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43689 } }
I20260621 07:56:32.865125 13320 sys_catalog.cc:564] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:32.866000 13326 sys_catalog.cc:455] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db651c9b4cf6444ebfe4cd428e381f19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db651c9b4cf6444ebfe4cd428e381f19" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43689 } } }
I20260621 07:56:32.866112 13326 sys_catalog.cc:458] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:32.866498 13327 sys_catalog.cc:455] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db651c9b4cf6444ebfe4cd428e381f19. Latest consensus state: current_term: 1 leader_uuid: "db651c9b4cf6444ebfe4cd428e381f19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db651c9b4cf6444ebfe4cd428e381f19" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43689 } } }
I20260621 07:56:32.866657 13327 sys_catalog.cc:458] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:32.867413 13334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:32.867993 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:32.854947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45210 (local address 127.11.223.190:43689)
0621 07:56:32.855307 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:32.855322 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:32.855358 (+ 36us) server_negotiation.cc:407] Connection header received
0621 07:56:32.855946 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:32.855961 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:32.856275 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:32.856630 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:32.857402 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:32.858409 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:32.859224 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:32.859651 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:32.859804 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:32.862350 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:32.862356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:32.862370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:32.862398 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:32.864255 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:32.864820 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:32.864826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:32.864831 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:32.864955 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:32.865388 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:32.865394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:32.865398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:32.865913 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:32.866690 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:32.866864 (+ 174us) server_negotiation.cc:299] Negotiation successful
0621 07:56:32.867285 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":117,"threads_started":1}
I20260621 07:56:32.870167 13334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:32.876209 13334 catalog_manager.cc:1347] Generated new cluster ID: 0391168ee03a415492e5c028f51fa560
I20260621 07:56:32.876267 13334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:32.883644 13334 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:32.884647 13334 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:32.902453 13334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: Generated new TSK 0
I20260621 07:56:32.903070 13334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:32.929087 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43689
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:33.029800 13346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:33.030004 13346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:33.030040 13346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:33.033957 13346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:33.034001 13346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:33.034037 13346 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:56:33.034066 13346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:33.038791 13346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:33.039665 13346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:33.040583 13346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:33.049084 13352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:33.049073 13351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:33.049072 13354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:33.049393 13346 server_base.cc:1034] running on GCE node
I20260621 07:56:33.049737 13346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:33.050303 13346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:33.051496 13346 hybrid_clock.cc:648] HybridClock initialized: now 1782028593051479 us; error 33 us; skew 500 ppm
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:33.053961 13346 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:33.054706 13346 webserver.cc:460] Webserver started at http://127.11.223.129:40593/ using document root <none> and password file <none>
I20260621 07:56:33.055199 13346 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:33.055277 13346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:33.055483 13346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:33.057230 13346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "ab2eea207f6a40f3b36c78914c269e7d"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "ee2bd964191a98a6edf2c9268ee0aab2"
server_key_iv: "1bf48b36e13cc11e7ebfac33c084debd"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.057721 13346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "ab2eea207f6a40f3b36c78914c269e7d"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "ee2bd964191a98a6edf2c9268ee0aab2"
server_key_iv: "1bf48b36e13cc11e7ebfac33c084debd"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.060946 13346 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:33.062992 13361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:33.064026 13346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:33.064145 13346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "ab2eea207f6a40f3b36c78914c269e7d"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "ee2bd964191a98a6edf2c9268ee0aab2"
server_key_iv: "1bf48b36e13cc11e7ebfac33c084debd"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.064236 13346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:33.081884 13346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:33.084846 13346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:33.085017 13346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:33.086261 13346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:33.086326 13346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:33.086417 13346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:33.086457 13346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:33.097373 13346 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:40677
I20260621 07:56:33.097432 13476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:40677 every 8 connection(s)
I20260621 07:56:33.099023 13346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:33.103205 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.086846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:35979 (local address 127.11.223.190:43689)
0621 07:56:33.086997 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.087001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.088237 (+ 1236us) server_negotiation.cc:407] Connection header received
0621 07:56:33.089964 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.089970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.090035 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.090131 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.091618 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.092527 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.094128 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.094530 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.094645 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.099369 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.099372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.099374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.099396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.101204 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.101751 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.101755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.101757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.101845 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.102172 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.102176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.102178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.102356 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.102453 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.102876 (+ 423us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.103010 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:56:33.103826 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.087679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.11.223.129:35979)
0621 07:56:33.088090 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.088135 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.089767 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.090322 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.090330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.090804 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:33.091446 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.091465 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.092674 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.092677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.094008 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.094018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.094708 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.094713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.094924 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.095514 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:33.095534 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:33.098611 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:33.101322 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.101327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.101340 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.101630 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.101934 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.101936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.101937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.102070 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.102461 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:33.102469 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:33.102671 (+ 202us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.102835 (+ 164us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.103101 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
I20260621 07:56:33.105007 13477 heartbeater.cc:347] Connected to a master server at 127.11.223.190:43689
I20260621 07:56:33.105150 13477 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:33.105584 13477 heartbeater.cc:511] Master 127.11.223.190:43689 requested a full tablet report, sending...
I20260621 07:56:33.107048 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13346
I20260621 07:56:33.107067 13273 ts_manager.cc:194] Registered new tserver with Master: ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677)
I20260621 07:56:33.107203 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:33.107514 12158 external_mini_cluster.cc:1428] Setting key c401f34e3330b28cc7d8e30ca4ca8098
I20260621 07:56:33.108525 13273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:35979
I20260621 07:56:33.114646 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.106908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.11.223.129:34555)
0621 07:56:33.107266 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.107281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.107392 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.107622 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.107626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.107816 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:33.107932 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.107938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.108877 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.108880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.109631 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.109641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.110284 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.110288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.110372 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.110864 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:33.110875 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:33.111182 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:33.113310 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.113312 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.113313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.113489 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.113761 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.113763 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.113765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.113831 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.114246 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:33.114248 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:33.114320 (+ 72us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.114395 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.114504 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":94,"threads_started":1}
I20260621 07:56:33.114857 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.106950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:34555 (local address 127.11.223.190:43689)
0621 07:56:33.107043 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.107046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.107360 (+ 314us) server_negotiation.cc:407] Connection header received
0621 07:56:33.107467 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.107470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.107506 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.107585 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.108039 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.108758 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.109751 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.110146 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.110275 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.111311 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.111317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.111323 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.111352 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.113200 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.113580 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.113586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.113590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.113673 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.113914 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.113918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.113922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.114156 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.114265 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.114495 (+ 230us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.114634 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:33.132709 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43689
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:33.237295 13486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:33.237573 13486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:33.237646 13486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:33.241963 13486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:33.242049 13486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:33.242120 13486 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:56:33.242198 13486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:33.247573 13486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:33.248559 13486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:33.249475 13486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:56:33.258633 13486 server_base.cc:1034] running on GCE node
W20260621 07:56:33.258683 13494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:33.258701 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:33.258831 13491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:33.259238 13486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:33.259801 13486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:33.260982 13486 hybrid_clock.cc:648] HybridClock initialized: now 1782028593260971 us; error 25 us; skew 500 ppm
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:33.263528 13486 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:33.264245 13486 webserver.cc:460] Webserver started at http://127.11.223.130:46127/ using document root <none> and password file <none>
I20260621 07:56:33.264783 13486 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:33.264842 13486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:33.264990 13486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:33.266947 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "8204082c65e14f4998a9bc885ca9806d"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "37915a4869ca9c7b2300658845843226"
server_key_iv: "6c5f786fb2dd2332d1b6bb7883257f50"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.267479 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "8204082c65e14f4998a9bc885ca9806d"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "37915a4869ca9c7b2300658845843226"
server_key_iv: "6c5f786fb2dd2332d1b6bb7883257f50"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.270907 13486 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:33.273200 13501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:33.274336 13486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:33.274431 13486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "8204082c65e14f4998a9bc885ca9806d"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "37915a4869ca9c7b2300658845843226"
server_key_iv: "6c5f786fb2dd2332d1b6bb7883257f50"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.274510 13486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:33.306020 13486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:33.308756 13486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:33.308928 13486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:33.310277 13486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:33.310352 13486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:33.310493 13486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:33.310632 13486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:33.321681 13486 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:43435
I20260621 07:56:33.321776 13616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:43435 every 8 connection(s)
I20260621 07:56:33.322567 13486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:33.324218 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.310962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:41625 (local address 127.11.223.190:43689)
0621 07:56:33.311102 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.311107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.311951 (+ 844us) server_negotiation.cc:407] Connection header received
0621 07:56:33.312925 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.312928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.312995 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.313097 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.314766 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.315601 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.317013 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.317247 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.317372 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.320390 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.320394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.320397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.320429 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.322131 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.322682 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.322687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.322689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.322778 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.323157 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.323161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.323163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.323506 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.323619 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.324011 (+ 392us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.324103 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:56:33.324975 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.311363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.11.223.130:41625)
0621 07:56:33.311789 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.311835 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.312722 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.313291 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.313299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.313766 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:33.314594 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.314634 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.315727 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.315732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.316876 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.316885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.317423 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.317429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.317679 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.318289 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:33.318312 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:33.320086 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:33.322291 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.322297 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.322318 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.322578 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.322869 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.322875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.322880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.323047 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.323616 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:33.323621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:33.323814 (+ 193us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.323960 (+ 146us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.324171 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":134,"threads_started":1}
I20260621 07:56:33.326041 13617 heartbeater.cc:347] Connected to a master server at 127.11.223.190:43689
I20260621 07:56:33.326184 13617 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:33.326628 13617 heartbeater.cc:511] Master 127.11.223.190:43689 requested a full tablet report, sending...
I20260621 07:56:33.327562 13274 ts_manager.cc:194] Registered new tserver with Master: 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435)
I20260621 07:56:33.327903 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13486
I20260621 07:56:33.328023 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:33.328303 13274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:41625
I20260621 07:56:33.328385 12158 external_mini_cluster.cc:1428] Setting key 1dbb706243e0b651092a4fa26fae180c
I20260621 07:56:33.335734 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.328202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:45287 (local address 127.11.223.190:43689)
0621 07:56:33.328349 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.328353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.328581 (+ 228us) server_negotiation.cc:407] Connection header received
0621 07:56:33.328772 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.328777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.328824 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.328900 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.329488 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.330144 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.330957 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.331344 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.331466 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.332652 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.332655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.332657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.332677 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.334084 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.334552 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.334554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.334555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.334639 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.334960 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.334963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.334964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.335155 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.335241 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.335555 (+ 314us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.335637 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:56:33.335919 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.328123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.11.223.130:45287)
0621 07:56:33.328500 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.328516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.328642 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.329022 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.329026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.329243 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:33.329354 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.329361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.330301 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.330305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.330830 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.330837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.331480 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.331485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.331600 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.332055 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:33.332074 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:33.332519 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:33.334200 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.334205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.334209 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.334457 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.334733 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.334736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.334738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.334842 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.335264 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:33.335268 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:33.335382 (+ 114us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.335508 (+ 126us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.335679 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:33.353193 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43689
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:33.456558 13626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:33.456773 13626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:33.456849 13626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:33.460741 13626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:33.460786 13626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:33.460839 13626 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:56:33.460911 13626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:33.465994 13626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:33.466877 13626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:33.467799 13626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:33.476481 13632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:33.476537 13631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:33.476634 13626 server_base.cc:1034] running on GCE node
W20260621 07:56:33.476538 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:33.477061 13626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:33.477684 13626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:33.478863 13626 hybrid_clock.cc:648] HybridClock initialized: now 1782028593478856 us; error 30 us; skew 500 ppm
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:33.481463 13626 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:33.482209 13626 webserver.cc:460] Webserver started at http://127.11.223.131:33003/ using document root <none> and password file <none>
I20260621 07:56:33.482702 13626 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:33.482765 13626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:33.482996 13626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:33.484870 13626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "ffab48f314254d6999e81e2dd3aed8d4"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "ca168a572b5f0bdc39e0c9c0e4ccbb17"
server_key_iv: "202548ef7c058d48c1207f5ffadc31db"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.485391 13626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "ffab48f314254d6999e81e2dd3aed8d4"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "ca168a572b5f0bdc39e0c9c0e4ccbb17"
server_key_iv: "202548ef7c058d48c1207f5ffadc31db"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.488844 13626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:33.491051 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:33.492163 13626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:33.492276 13626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "ffab48f314254d6999e81e2dd3aed8d4"
format_stamp: "Formatted at 2026-06-21 07:56:33 on dist-test-slave-0pn3"
server_key: "ca168a572b5f0bdc39e0c9c0e4ccbb17"
server_key_iv: "202548ef7c058d48c1207f5ffadc31db"
server_key_version: "encryptionkey@0"
I20260621 07:56:33.492425 13626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:33.506844 13626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:33.509785 13626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:33.509972 13626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:33.511322 13626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:33.511373 13626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:33.511411 13626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:33.511471 13626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:33.522109 13626 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:36767
I20260621 07:56:33.522182 13756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:36767 every 8 connection(s)
I20260621 07:56:33.522922 13626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:33.525553 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.511943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:48639 (local address 127.11.223.190:43689)
0621 07:56:33.512065 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.512068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.513582 (+ 1514us) server_negotiation.cc:407] Connection header received
0621 07:56:33.514619 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.514622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.514678 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.514765 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.516360 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.517203 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.518995 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.519200 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.519283 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.522031 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.522034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.522037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.522082 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.523662 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.524112 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.524117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.524119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.524194 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.524541 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.524546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.524548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.524737 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.524847 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.525283 (+ 436us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.525391 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:33.526297 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.512964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.11.223.131:48639)
0621 07:56:33.513431 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.513488 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.514425 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.514982 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.514993 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.515467 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:33.516157 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.516178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.517364 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.517371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.518865 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.518875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.519362 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.519367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.519563 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.520304 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:33.520342 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:33.521869 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:33.523763 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.523770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.523782 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.524013 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.524292 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.524295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.524296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.524448 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.524844 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:33.524850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:33.525059 (+ 209us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.525224 (+ 165us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.525505 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":129,"threads_started":1}
I20260621 07:56:33.527309 13757 heartbeater.cc:347] Connected to a master server at 127.11.223.190:43689
I20260621 07:56:33.527465 13757 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:33.527917 13757 heartbeater.cc:511] Master 127.11.223.190:43689 requested a full tablet report, sending...
I20260621 07:56:33.528401 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13626
I20260621 07:56:33.528525 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:33.528792 12158 external_mini_cluster.cc:1428] Setting key e03ca07d017521f613cae3eacee6913d
I20260621 07:56:33.528928 13274 ts_manager.cc:194] Registered new tserver with Master: ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131:36767)
I20260621 07:56:33.529644 13274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:48639
I20260621 07:56:33.530804 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:56:33.537387 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.529486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:50181 (local address 127.11.223.190:43689)
0621 07:56:33.529575 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.529578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.530016 (+ 438us) server_negotiation.cc:407] Connection header received
0621 07:56:33.530135 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.530138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.530182 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.530264 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.531846 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.532397 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.533188 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.533418 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.533521 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.534485 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.534488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.534491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.534513 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.535832 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.536236 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.536240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.536242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.536365 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.536626 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.536629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.536631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.536797 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.536899 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.537147 (+ 248us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.537245 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260621 07:56:33.537406 13763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.529413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.11.223.131:50181)
0621 07:56:33.529811 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.529945 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.530052 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.531490 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.531492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.531642 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:33.531721 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.531725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.532521 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.532525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.533073 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.533080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.533532 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.533535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.533626 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.534059 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:33.534069 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:33.534352 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:33.535940 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.535945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.535947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.536135 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.536442 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:33.536445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:33.536447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:33.536531 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:33.536890 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:33.536893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:33.536988 (+ 95us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.537094 (+ 106us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.537219 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":104,"threads_started":1}
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 07:56:33 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:33.556416 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.548709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.11.223.190:43689)
0621 07:56:33.548845 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.548849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.548860 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:33.548891 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.548894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.548934 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.549016 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:33.549700 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.550196 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.551014 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.551254 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.551359 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.553607 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:33.553611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:33.553613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:33.553665 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:33.554811 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.555242 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.555247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.555248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.555327 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:33.555601 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:33.555604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:33.555606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:33.555769 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:33.555899 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.556130 (+ 231us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.556238 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:33.558838 13274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:33.562083 13274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:33.575628 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.569915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43435 (local address 127.0.0.1:32782)
0621 07:56:33.570676 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.570722 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.570845 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.571463 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.571469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.571491 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.571607 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.571613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.572871 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.572876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.574187 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.574197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.575106 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.575110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.575213 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.575250 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.575373 (+ 123us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.575484 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260621 07:56:33.575671 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.570027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32782 (local address 127.11.223.130:43435)
0621 07:56:33.570455 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.570465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.570990 (+ 525us) server_negotiation.cc:407] Connection header received
0621 07:56:33.571079 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.571085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.571296 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.571463 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.571742 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.572762 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.574310 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.574981 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.575115 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.575253 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.575412 (+ 159us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.575504 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":141,"threads_started":1}
I20260621 07:56:33.576026 13775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.570632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36767 (local address 127.0.0.1:35914)
0621 07:56:33.571303 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.571316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.571395 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.572054 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.572058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.572075 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.572149 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.572154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.573434 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.573437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.574636 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.574644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.575631 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.575636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.575751 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.575792 (+ 41us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.575855 (+ 63us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.575909 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":20,"thread_start_us":90,"threads_started":1}
I20260621 07:56:33.576529 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.571000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.11.223.129:40677)
0621 07:56:33.571220 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.571233 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.571257 (+ 24us) server_negotiation.cc:407] Connection header received
0621 07:56:33.571335 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.571343 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.571530 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.571676 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.571870 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.573044 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.574649 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.575294 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.575423 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.575534 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.576219 (+ 685us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.576306 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260621 07:56:33.577003 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.570770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.11.223.131:36767)
0621 07:56:33.571638 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.571649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.571668 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:56:33.571734 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.571739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.571916 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.572061 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.572268 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.573318 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.574753 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.575527 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.575677 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.575807 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.575897 (+ 90us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.576857 (+ 960us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":79,"threads_started":1}
I20260621 07:56:33.576236 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.570004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:40677 (local address 127.0.0.1:56366)
0621 07:56:33.571138 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.571151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.571227 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.571667 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.571672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.571692 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.571777 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.571786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.573194 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.573197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.574505 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.574515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.575844 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.575848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.575948 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.575969 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.576044 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.576113 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"mutex_wait_us":46,"thread_start_us":87,"threads_started":1}
I20260621 07:56:33.578975 13551 tablet_service.cc:1467] Processing CreateTablet for tablet e311ef7351ac4026b99737d95c92418b (DEFAULT_TABLE table=test-table [id=ddf5122f980348a78265377ee67e4b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:33.579615 13406 tablet_service.cc:1467] Processing CreateTablet for tablet e311ef7351ac4026b99737d95c92418b (DEFAULT_TABLE table=test-table [id=ddf5122f980348a78265377ee67e4b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:33.580051 13551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e311ef7351ac4026b99737d95c92418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:33.580603 13406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e311ef7351ac4026b99737d95c92418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:33.586434 13691 tablet_service.cc:1467] Processing CreateTablet for tablet e311ef7351ac4026b99737d95c92418b (DEFAULT_TABLE table=test-table [id=ddf5122f980348a78265377ee67e4b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:33.587203 13691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e311ef7351ac4026b99737d95c92418b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:33.587821 13780 tablet_bootstrap.cc:492] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Bootstrap starting.
I20260621 07:56:33.588217 13781 tablet_bootstrap.cc:492] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Bootstrap starting.
I20260621 07:56:33.592447 13781 tablet_bootstrap.cc:654] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:33.592617 13780 tablet_bootstrap.cc:654] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:33.593245 13781 log.cc:824] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:33.593444 13780 log.cc:824] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:33.594347 13784 tablet_bootstrap.cc:492] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Bootstrap starting.
I20260621 07:56:33.595207 13781 tablet_bootstrap.cc:492] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: No bootstrap required, opened a new log
I20260621 07:56:33.595392 13781 ts_tablet_manager.cc:1397] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260621 07:56:33.596118 13784 tablet_bootstrap.cc:654] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:33.596199 13780 tablet_bootstrap.cc:492] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: No bootstrap required, opened a new log
I20260621 07:56:33.596403 13780 ts_tablet_manager.cc:1397] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260621 07:56:33.596879 13784 log.cc:824] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:33.598676 13784 tablet_bootstrap.cc:492] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: No bootstrap required, opened a new log
I20260621 07:56:33.598867 13784 ts_tablet_manager.cc:1397] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260621 07:56:33.599305 13781 raft_consensus.cc:348] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.599553 13781 raft_consensus.cc:374] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:33.599647 13781 raft_consensus.cc:729] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8204082c65e14f4998a9bc885ca9806d, State: Initialized, Role: FOLLOWER
I20260621 07:56:33.600073 13781 consensus_queue.cc:260] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.600368 13780 raft_consensus.cc:348] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.600651 13780 raft_consensus.cc:374] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:33.600710 13617 heartbeater.cc:503] Master 127.11.223.190:43689 was elected leader, sending a full tablet report...
I20260621 07:56:33.600735 13780 raft_consensus.cc:729] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2eea207f6a40f3b36c78914c269e7d, State: Initialized, Role: FOLLOWER
I20260621 07:56:33.601071 13781 ts_tablet_manager.cc:1428] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260621 07:56:33.601173 13780 consensus_queue.cc:260] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.601783 13477 heartbeater.cc:503] Master 127.11.223.190:43689 was elected leader, sending a full tablet report...
I20260621 07:56:33.602094 13780 ts_tablet_manager.cc:1428] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260621 07:56:33.602721 13784 raft_consensus.cc:348] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.602964 13784 raft_consensus.cc:374] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:33.603057 13784 raft_consensus.cc:729] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffab48f314254d6999e81e2dd3aed8d4, State: Initialized, Role: FOLLOWER
I20260621 07:56:33.603493 13784 consensus_queue.cc:260] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.604370 13757 heartbeater.cc:503] Master 127.11.223.190:43689 was elected leader, sending a full tablet report...
I20260621 07:56:33.604408 13784 ts_tablet_manager.cc:1428] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260621 07:56:33.755007 13788 raft_consensus.cc:482] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:33.755205 13788 raft_consensus.cc:504] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.756255 13788 leader_election.cc:288] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435), ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677)
I20260621 07:56:33.760027 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.756590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:58155 (local address 127.11.223.129:40677)
0621 07:56:33.756873 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.756877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.756890 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:56:33.757012 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.757014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.757054 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.757118 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.757388 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.757884 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.758990 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.759451 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.759594 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.759650 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.759845 (+ 195us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.759908 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260621 07:56:33.760027 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.756517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:40677 (local address 127.11.223.131:58155)
0621 07:56:33.756789 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.756807 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.756898 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.757145 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.757148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.757162 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.757264 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.757269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.757995 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.757999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.758868 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.758877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.759594 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.759598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.759716 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.759733 (+ 17us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.759830 (+ 97us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.759899 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":54}
I20260621 07:56:33.760588 13789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.756517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43435 (local address 127.11.223.131:39751)
0621 07:56:33.756895 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.756907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.756994 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.757390 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.757393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.757410 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.757854 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.757860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.758580 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.758585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.759448 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.759456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.760093 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.760097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.760209 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.760234 (+ 25us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.760374 (+ 140us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.760439 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":85,"thread_start_us":132,"threads_started":1}
I20260621 07:56:33.760550 13786 raft_consensus.cc:482] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:33.760589 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.756603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:39751 (local address 127.11.223.130:43435)
0621 07:56:33.756749 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.756754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.757202 (+ 448us) server_negotiation.cc:407] Connection header received
0621 07:56:33.757246 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.757248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.757282 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.757372 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.757971 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.758468 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.759580 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.759970 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.760092 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.760142 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.760399 (+ 257us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.760483 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:56:33.760707 13428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e311ef7351ac4026b99737d95c92418b" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2eea207f6a40f3b36c78914c269e7d" is_pre_election: true
I20260621 07:56:33.760723 13786 raft_consensus.cc:504] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.760982 13428 raft_consensus.cc:2454] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 0.
I20260621 07:56:33.761564 13645 leader_election.cc:302] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab2eea207f6a40f3b36c78914c269e7d, ffab48f314254d6999e81e2dd3aed8d4; no voters:
I20260621 07:56:33.761693 13571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e311ef7351ac4026b99737d95c92418b" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8204082c65e14f4998a9bc885ca9806d" is_pre_election: true
I20260621 07:56:33.761806 13786 leader_election.cc:288] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677), ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131:36767)
I20260621 07:56:33.761862 13571 raft_consensus.cc:2454] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 0.
I20260621 07:56:33.761865 13788 raft_consensus.cc:2781] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:33.761945 13788 raft_consensus.cc:482] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:33.761992 13788 raft_consensus.cc:3037] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:33.763540 13788 raft_consensus.cc:504] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.764181 13571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e311ef7351ac4026b99737d95c92418b" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8204082c65e14f4998a9bc885ca9806d"
I20260621 07:56:33.764252 13788 leader_election.cc:288] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 election: Requested vote from peers 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435), ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677)
I20260621 07:56:33.764343 13571 raft_consensus.cc:3037] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:33.765275 13571 raft_consensus.cc:2454] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 1.
I20260621 07:56:33.765555 13428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e311ef7351ac4026b99737d95c92418b" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2eea207f6a40f3b36c78914c269e7d"
I20260621 07:56:33.765578 13644 leader_election.cc:302] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8204082c65e14f4998a9bc885ca9806d, ffab48f314254d6999e81e2dd3aed8d4; no voters:
I20260621 07:56:33.765672 13428 raft_consensus.cc:3037] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:33.765743 13788 raft_consensus.cc:2781] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:33.765936 13788 raft_consensus.cc:686] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 1 LEADER]: Becoming Leader. State: Replica: ffab48f314254d6999e81e2dd3aed8d4, State: Running, Role: LEADER
I20260621 07:56:33.766235 13788 consensus_queue.cc:237] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.766408 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.761972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:40677 (local address 127.11.223.130:40229)
0621 07:56:33.762105 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.762123 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.762213 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.762875 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.762878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.762900 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.762995 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.763000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.763910 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.763913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.765084 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.765093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.766022 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.766025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.766120 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.766138 (+ 18us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.766203 (+ 65us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.766287 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260621 07:56:33.766906 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.762303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:53505 (local address 127.11.223.131:36767)
0621 07:56:33.762432 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.762436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.762757 (+ 321us) server_negotiation.cc:407] Connection header received
0621 07:56:33.762917 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.762920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.762963 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.763042 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.763622 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.764303 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.765711 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.766231 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.766358 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.766396 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.766747 (+ 351us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.766812 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:56:33.766896 13428 raft_consensus.cc:2454] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 1.
I20260621 07:56:33.766901 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.762466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:40229 (local address 127.11.223.129:40677)
0621 07:56:33.762560 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.762564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.762577 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:56:33.762626 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.762628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.762674 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.762761 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:33.763131 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.763790 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.765216 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.765809 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.765919 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.765966 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.766653 (+ 687us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.766725 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260621 07:56:33.767211 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.762234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36767 (local address 127.11.223.130:53505)
0621 07:56:33.762676 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:33.762690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:33.762830 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:33.763057 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:33.763059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:33.763086 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:33.763511 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.763517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.764437 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.764440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.765572 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:33.765582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.766513 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.766516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:33.766604 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.766623 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 07:56:33.767013 (+ 390us) client_negotiation.cc:240] Negotiation successful
0621 07:56:33.767085 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260621 07:56:33.767719 13711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e311ef7351ac4026b99737d95c92418b" candidate_uuid: "8204082c65e14f4998a9bc885ca9806d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffab48f314254d6999e81e2dd3aed8d4" is_pre_election: true
I20260621 07:56:33.767758 13428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e311ef7351ac4026b99737d95c92418b" candidate_uuid: "8204082c65e14f4998a9bc885ca9806d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2eea207f6a40f3b36c78914c269e7d" is_pre_election: true
I20260621 07:56:33.767936 13428 raft_consensus.cc:2379] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8204082c65e14f4998a9bc885ca9806d in current term 1: Already voted for candidate ffab48f314254d6999e81e2dd3aed8d4 in this term.
I20260621 07:56:33.768541 13505 leader_election.cc:302] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8204082c65e14f4998a9bc885ca9806d; no voters: ab2eea207f6a40f3b36c78914c269e7d, ffab48f314254d6999e81e2dd3aed8d4
I20260621 07:56:33.768767 13786 raft_consensus.cc:2726] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260621 07:56:33.769820 13274 catalog_manager.cc:5495] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131). New cstate: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } health_report { overall_health: HEALTHY } } }
W20260621 07:56:33.773789 13758 tablet.cc:2369] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:33.810108 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:33.806581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.11.223.131:36767)
0621 07:56:33.806685 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:33.806691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:33.806739 (+ 48us) server_negotiation.cc:407] Connection header received
0621 07:56:33.806943 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:33.806952 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:33.806996 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:33.807086 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:33.807311 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.807734 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.808694 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:33.808908 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:33.809012 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:33.809347 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:33.809724 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:33.809848 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:33.809924 (+ 76us) server_negotiation.cc:299] Negotiation successful
0621 07:56:33.809968 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 07:56:33.814862 13428 raft_consensus.cc:1261] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Refusing update from remote peer ffab48f314254d6999e81e2dd3aed8d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:33.814911 13571 raft_consensus.cc:1261] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Refusing update from remote peer ffab48f314254d6999e81e2dd3aed8d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:33.815440 13791 consensus_queue.cc:1035] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:33.815610 13788 consensus_queue.cc:1035] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:33.821444 13794 mvcc.cc:205] Tried to move back new op lower bound from 7299189120259973120 to 7299189120070205440. Current Snapshot: MvccSnapshot[applied={T|T < 7299189120259973120}]
I20260621 07:56:33.824079 13796 mvcc.cc:205] Tried to move back new op lower bound from 7299189120259973120 to 7299189120070205440. Current Snapshot: MvccSnapshot[applied={T|T < 7299189120259973120}]
I20260621 07:56:33.824247 13795 mvcc.cc:205] Tried to move back new op lower bound from 7299189120259973120 to 7299189120070205440. Current Snapshot: MvccSnapshot[applied={T|T < 7299189120259973120}]
I20260621 07:56:33.832058 13273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:33.832198 13273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:33.834426 13273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: Sending DeleteTablet for 3 replicas of tablet e311ef7351ac4026b99737d95c92418b
I20260621 07:56:33.835068 13551 tablet_service.cc:1514] Processing DeleteTablet for tablet e311ef7351ac4026b99737d95c92418b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:33 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:32782
I20260621 07:56:33.835119 13406 tablet_service.cc:1514] Processing DeleteTablet for tablet e311ef7351ac4026b99737d95c92418b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:33 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:56366
I20260621 07:56:33.835619 13691 tablet_service.cc:1514] Processing DeleteTablet for tablet e311ef7351ac4026b99737d95c92418b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:33 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35914
I20260621 07:56:33.835749 13803 tablet_replica.cc:330] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: stopping tablet replica
I20260621 07:56:33.835749 13273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260621 07:56:33.836023 13803 raft_consensus.cc:2229] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:33.835486 13804 tablet_replica.cc:330] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: stopping tablet replica
W20260621 07:56:33.836112 13273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:33.836352 13804 raft_consensus.cc:2229] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:33.836444 13803 raft_consensus.cc:2258] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:33.836776 13804 raft_consensus.cc:2258] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:33.837278 13805 tablet_replica.cc:330] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: stopping tablet replica
I20260621 07:56:33.838424 13803 ts_tablet_manager.cc:1905] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:33.839105 13804 ts_tablet_manager.cc:1905] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:33.839108 13805 raft_consensus.cc:2229] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:56:33.839409 13805 raft_consensus.cc:2258] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:33.840771 13406 tablet_service.cc:1467] Processing CreateTablet for tablet 1562a4292b604d55b912d19657f3e523 (DEFAULT_TABLE table=test-table [id=8275c6e05667433eaefae47452099b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:33.841061 13406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1562a4292b604d55b912d19657f3e523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:33.841228 13803 ts_tablet_manager.cc:1918] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:33.841380 13803 log.cc:1196] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/e311ef7351ac4026b99737d95c92418b
I20260621 07:56:33.841744 13803 ts_tablet_manager.cc:1939] T e311ef7351ac4026b99737d95c92418b P 8204082c65e14f4998a9bc885ca9806d: Deleting consensus metadata
I20260621 07:56:33.842171 13804 ts_tablet_manager.cc:1918] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:33.842341 13804 log.cc:1196] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/e311ef7351ac4026b99737d95c92418b
I20260621 07:56:33.842706 13804 ts_tablet_manager.cc:1939] T e311ef7351ac4026b99737d95c92418b P ab2eea207f6a40f3b36c78914c269e7d: Deleting consensus metadata
I20260621 07:56:33.842900 13260 catalog_manager.cc:4841] TS 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435): tablet e311ef7351ac4026b99737d95c92418b (table test-table [id=ddf5122f980348a78265377ee67e4b49]) successfully deleted
I20260621 07:56:33.843283 13780 tablet_bootstrap.cc:492] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Bootstrap starting.
I20260621 07:56:33.843936 13261 catalog_manager.cc:4841] TS ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677): tablet e311ef7351ac4026b99737d95c92418b (table test-table [id=ddf5122f980348a78265377ee67e4b49]) successfully deleted
I20260621 07:56:33.844210 13780 tablet_bootstrap.cc:654] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:33.844763 13551 tablet_service.cc:1467] Processing CreateTablet for tablet 1562a4292b604d55b912d19657f3e523 (DEFAULT_TABLE table=test-table [id=8275c6e05667433eaefae47452099b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:33.844967 13551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1562a4292b604d55b912d19657f3e523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:33.845319 13805 ts_tablet_manager.cc:1905] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:33.847030 13781 tablet_bootstrap.cc:492] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Bootstrap starting.
I20260621 07:56:33.847757 13805 ts_tablet_manager.cc:1918] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:33.847864 13805 log.cc:1196] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/e311ef7351ac4026b99737d95c92418b
I20260621 07:56:33.848150 13805 ts_tablet_manager.cc:1939] T e311ef7351ac4026b99737d95c92418b P ffab48f314254d6999e81e2dd3aed8d4: Deleting consensus metadata
I20260621 07:56:33.848265 13691 tablet_service.cc:1467] Processing CreateTablet for tablet 1562a4292b604d55b912d19657f3e523 (DEFAULT_TABLE table=test-table [id=8275c6e05667433eaefae47452099b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:33.848531 13691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1562a4292b604d55b912d19657f3e523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:33.849311 13261 catalog_manager.cc:4841] TS ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131:36767): tablet e311ef7351ac4026b99737d95c92418b (table test-table [id=ddf5122f980348a78265377ee67e4b49]) successfully deleted
I20260621 07:56:33.850061 13780 tablet_bootstrap.cc:492] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: No bootstrap required, opened a new log
I20260621 07:56:33.850150 13780 ts_tablet_manager.cc:1397] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260621 07:56:33.850544 13780 raft_consensus.cc:348] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.850646 13780 raft_consensus.cc:374] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:33.850673 13780 raft_consensus.cc:729] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2eea207f6a40f3b36c78914c269e7d, State: Initialized, Role: FOLLOWER
I20260621 07:56:33.850773 13780 consensus_queue.cc:260] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.850975 13780 ts_tablet_manager.cc:1428] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:56:33.851132 13784 tablet_bootstrap.cc:492] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Bootstrap starting.
I20260621 07:56:33.851970 13784 tablet_bootstrap.cc:654] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:33.853056 13784 tablet_bootstrap.cc:492] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: No bootstrap required, opened a new log
I20260621 07:56:33.853125 13784 ts_tablet_manager.cc:1397] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:33.853341 13781 tablet_bootstrap.cc:654] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:33.853507 13784 raft_consensus.cc:348] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.853596 13784 raft_consensus.cc:374] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:33.853622 13784 raft_consensus.cc:729] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffab48f314254d6999e81e2dd3aed8d4, State: Initialized, Role: FOLLOWER
I20260621 07:56:33.853716 13784 consensus_queue.cc:260] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.853909 13784 ts_tablet_manager.cc:1428] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:33.854914 13781 tablet_bootstrap.cc:492] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: No bootstrap required, opened a new log
I20260621 07:56:33.855002 13781 ts_tablet_manager.cc:1397] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260621 07:56:33.855427 13781 raft_consensus.cc:348] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.855522 13781 raft_consensus.cc:374] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:33.855548 13781 raft_consensus.cc:729] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8204082c65e14f4998a9bc885ca9806d, State: Initialized, Role: FOLLOWER
I20260621 07:56:33.855643 13781 consensus_queue.cc:260] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:33.855844 13781 ts_tablet_manager.cc:1428] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:56:34.054658 13787 raft_consensus.cc:482] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:34.054819 13787 raft_consensus.cc:504] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.055830 13787 leader_election.cc:288] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435), ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131:36767)
I20260621 07:56:34.060462 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.056064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:55259 (local address 127.11.223.130:43435)
0621 07:56:34.056197 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.056202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.056371 (+ 169us) server_negotiation.cc:407] Connection header received
0621 07:56:34.056574 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.056577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.056614 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.056684 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:34.056949 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.057592 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.059183 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.059743 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.059854 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.059901 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.060262 (+ 361us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.060343 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260621 07:56:34.060462 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.056294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:42461 (local address 127.11.223.131:36767)
0621 07:56:34.056416 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.056420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.056495 (+ 75us) server_negotiation.cc:407] Connection header received
0621 07:56:34.056639 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.056642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.056690 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.056768 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:34.057188 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.057798 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.059183 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.059743 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.059854 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.059901 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.060256 (+ 355us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.060350 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:56:34.060515 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.056030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36767 (local address 127.11.223.129:42461)
0621 07:56:34.056403 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:34.056416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:34.056543 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:34.056805 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:34.056809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:34.057006 (+ 197us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:34.057091 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.057096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.057881 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.057884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.059080 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.059088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.059875 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.059878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.060078 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.060127 (+ 49us) client_negotiation.cc:769] Sending connection context
0621 07:56:34.060225 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:56:34.060347 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":176,"thread_start_us":109,"threads_started":1}
I20260621 07:56:34.060515 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.056002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43435 (local address 127.11.223.129:55259)
0621 07:56:34.056259 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:34.056277 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:34.056452 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:34.056705 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:34.056707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:34.056721 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:34.056842 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.056848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.057689 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.057691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.059057 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.059067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.059875 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.059878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.060075 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.060128 (+ 53us) client_negotiation.cc:769] Sending connection context
0621 07:56:34.060227 (+ 99us) client_negotiation.cc:240] Negotiation successful
0621 07:56:34.060347 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":127,"thread_start_us":76,"threads_started":1}
I20260621 07:56:34.060993 13571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1562a4292b604d55b912d19657f3e523" candidate_uuid: "ab2eea207f6a40f3b36c78914c269e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8204082c65e14f4998a9bc885ca9806d" is_pre_election: true
I20260621 07:56:34.061087 13711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1562a4292b604d55b912d19657f3e523" candidate_uuid: "ab2eea207f6a40f3b36c78914c269e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffab48f314254d6999e81e2dd3aed8d4" is_pre_election: true
I20260621 07:56:34.061154 13571 raft_consensus.cc:2454] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab2eea207f6a40f3b36c78914c269e7d in term 0.
I20260621 07:56:34.061249 13711 raft_consensus.cc:2454] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab2eea207f6a40f3b36c78914c269e7d in term 0.
I20260621 07:56:34.061568 13365 leader_election.cc:302] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab2eea207f6a40f3b36c78914c269e7d, ffab48f314254d6999e81e2dd3aed8d4; no voters:
I20260621 07:56:34.061805 13787 raft_consensus.cc:2781] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:34.061903 13787 raft_consensus.cc:482] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:34.061954 13787 raft_consensus.cc:3037] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:34.062877 13787 raft_consensus.cc:504] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.063256 13787 leader_election.cc:288] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [CANDIDATE]: Term 1 election: Requested vote from peers 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435), ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131:36767)
I20260621 07:56:34.063563 13711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1562a4292b604d55b912d19657f3e523" candidate_uuid: "ab2eea207f6a40f3b36c78914c269e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffab48f314254d6999e81e2dd3aed8d4"
I20260621 07:56:34.063689 13711 raft_consensus.cc:3037] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:34.064457 13571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1562a4292b604d55b912d19657f3e523" candidate_uuid: "ab2eea207f6a40f3b36c78914c269e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8204082c65e14f4998a9bc885ca9806d"
I20260621 07:56:34.064554 13571 raft_consensus.cc:3037] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:34.064761 13711 raft_consensus.cc:2454] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab2eea207f6a40f3b36c78914c269e7d in term 1.
I20260621 07:56:34.065100 13365 leader_election.cc:302] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab2eea207f6a40f3b36c78914c269e7d, ffab48f314254d6999e81e2dd3aed8d4; no voters:
I20260621 07:56:34.065277 13787 raft_consensus.cc:2781] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:34.065490 13787 raft_consensus.cc:686] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 1 LEADER]: Becoming Leader. State: Replica: ab2eea207f6a40f3b36c78914c269e7d, State: Running, Role: LEADER
I20260621 07:56:34.065744 13571 raft_consensus.cc:2454] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab2eea207f6a40f3b36c78914c269e7d in term 1.
I20260621 07:56:34.065724 13787 consensus_queue.cc:237] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.067973 13273 catalog_manager.cc:5495] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d reported cstate change: term changed from 0 to 1, leader changed from <none> to ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129). New cstate: current_term: 1 leader_uuid: "ab2eea207f6a40f3b36c78914c269e7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } health_report { overall_health: UNKNOWN } } }
W20260621 07:56:34.074311 13618 tablet.cc:2369] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:56:34.100435 13478 tablet.cc:2369] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 21 07:56:34 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028592, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:34.121263 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.113881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43689 (local address 127.0.0.1:45230)
0621 07:56:34.114165 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:34.114178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:34.114261 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:34.114476 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:34.114478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:34.114802 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:34.114878 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.114883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.115512 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.115514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.115999 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.116004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.116460 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.116462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.116520 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.116988 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:34.116998 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:34.118289 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:34.119806 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:34.119814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:34.119817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:34.120031 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:34.120309 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:34.120341 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:34.120346 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:34.120446 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:34.120798 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:34.120802 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:34.120934 (+ 132us) client_negotiation.cc:769] Sending connection context
0621 07:56:34.121022 (+ 88us) client_negotiation.cc:240] Negotiation successful
0621 07:56:34.121131 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
I20260621 07:56:34.121428 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.113979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.11.223.190:43689)
0621 07:56:34.114225 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.114232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.114245 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:56:34.114338 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.114340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.114376 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.114449 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:34.114963 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.115434 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.116082 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.116287 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.116455 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.118405 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:34.118408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:34.118410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:34.118430 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:34.119702 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:34.120130 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:34.120136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:34.120139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:34.120232 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:34.120528 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:34.120530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:34.120532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:34.120679 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:34.120749 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.121075 (+ 326us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.121216 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":193,"thread_start_us":111,"threads_started":1}
I20260621 07:56:34.124578 13273 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:45230:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260621 07:56:34.125190 13273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:34.129733 13551 tablet_service.cc:1467] Processing CreateTablet for tablet 9714f9783bc940db8ea399b84e7178d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8266aa6dda654c60a4346891917d7b96]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:56:34.129698 13406 tablet_service.cc:1467] Processing CreateTablet for tablet 9714f9783bc940db8ea399b84e7178d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8266aa6dda654c60a4346891917d7b96]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:56:34.130059 13551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9714f9783bc940db8ea399b84e7178d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:34.130059 13406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9714f9783bc940db8ea399b84e7178d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:34.130033 13691 tablet_service.cc:1467] Processing CreateTablet for tablet 9714f9783bc940db8ea399b84e7178d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8266aa6dda654c60a4346891917d7b96]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:56:34.130367 13691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9714f9783bc940db8ea399b84e7178d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:34.132422 13784 tablet_bootstrap.cc:492] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: Bootstrap starting.
I20260621 07:56:34.132861 13780 tablet_bootstrap.cc:492] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: Bootstrap starting.
I20260621 07:56:34.133033 13781 tablet_bootstrap.cc:492] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: Bootstrap starting.
I20260621 07:56:34.133360 13784 tablet_bootstrap.cc:654] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:34.133728 13780 tablet_bootstrap.cc:654] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:34.133927 13781 tablet_bootstrap.cc:654] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:34.134688 13784 tablet_bootstrap.cc:492] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: No bootstrap required, opened a new log
I20260621 07:56:34.134820 13784 ts_tablet_manager.cc:1397] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260621 07:56:34.135033 13780 tablet_bootstrap.cc:492] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: No bootstrap required, opened a new log
I20260621 07:56:34.135099 13780 ts_tablet_manager.cc:1397] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260621 07:56:34.135273 13784 raft_consensus.cc:348] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.135381 13784 raft_consensus.cc:374] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:34.135424 13784 raft_consensus.cc:729] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffab48f314254d6999e81e2dd3aed8d4, State: Initialized, Role: FOLLOWER
I20260621 07:56:34.135502 13780 raft_consensus.cc:348] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.135617 13780 raft_consensus.cc:374] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:34.135550 13784 consensus_queue.cc:260] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.135643 13780 raft_consensus.cc:729] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2eea207f6a40f3b36c78914c269e7d, State: Initialized, Role: FOLLOWER
I20260621 07:56:34.135792 13780 consensus_queue.cc:260] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.135788 13788 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.135852 13784 ts_tablet_manager.cc:1428] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260621 07:56:34.135926 13788 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.136031 13780 ts_tablet_manager.cc:1428] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:34.136189 13787 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.136297 13787 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.137187 13781 tablet_bootstrap.cc:492] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: No bootstrap required, opened a new log
I20260621 07:56:34.137275 13781 ts_tablet_manager.cc:1397] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260621 07:56:34.137703 13781 raft_consensus.cc:348] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.137809 13781 raft_consensus.cc:374] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:34.137842 13781 raft_consensus.cc:729] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8204082c65e14f4998a9bc885ca9806d, State: Initialized, Role: FOLLOWER
I20260621 07:56:34.137959 13781 consensus_queue.cc:260] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.138233 13781 ts_tablet_manager.cc:1428] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260621 07:56:34.138201 13786 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.138336 13786 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.168959 13788 raft_consensus.cc:482] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:34.169057 13788 raft_consensus.cc:504] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.169353 13788 leader_election.cc:288] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677), 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435)
I20260621 07:56:34.169708 13428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9714f9783bc940db8ea399b84e7178d6" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2eea207f6a40f3b36c78914c269e7d" is_pre_election: true
I20260621 07:56:34.169713 13571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9714f9783bc940db8ea399b84e7178d6" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8204082c65e14f4998a9bc885ca9806d" is_pre_election: true
I20260621 07:56:34.169847 13428 raft_consensus.cc:2454] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 0.
I20260621 07:56:34.169874 13571 raft_consensus.cc:2454] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 0.
I20260621 07:56:34.170173 13645 leader_election.cc:302] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab2eea207f6a40f3b36c78914c269e7d, ffab48f314254d6999e81e2dd3aed8d4; no voters:
I20260621 07:56:34.170356 13788 raft_consensus.cc:2781] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:34.170413 13788 raft_consensus.cc:482] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:34.170441 13788 raft_consensus.cc:3037] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:34.171348 13788 raft_consensus.cc:504] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.171700 13788 leader_election.cc:288] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 election: Requested vote from peers ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677), 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435)
I20260621 07:56:34.171960 13428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9714f9783bc940db8ea399b84e7178d6" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2eea207f6a40f3b36c78914c269e7d"
I20260621 07:56:34.172087 13428 raft_consensus.cc:3037] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:34.172080 13571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9714f9783bc940db8ea399b84e7178d6" candidate_uuid: "ffab48f314254d6999e81e2dd3aed8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8204082c65e14f4998a9bc885ca9806d"
I20260621 07:56:34.172189 13571 raft_consensus.cc:3037] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:34.173262 13571 raft_consensus.cc:2454] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 1.
I20260621 07:56:34.173269 13428 raft_consensus.cc:2454] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffab48f314254d6999e81e2dd3aed8d4 in term 1.
I20260621 07:56:34.173528 13644 leader_election.cc:302] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8204082c65e14f4998a9bc885ca9806d, ffab48f314254d6999e81e2dd3aed8d4; no voters:
I20260621 07:56:34.173691 13788 raft_consensus.cc:2781] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:34.173815 13788 raft_consensus.cc:686] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 LEADER]: Becoming Leader. State: Replica: ffab48f314254d6999e81e2dd3aed8d4, State: Running, Role: LEADER
I20260621 07:56:34.173934 13788 consensus_queue.cc:237] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } }
I20260621 07:56:34.174504 13791 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: TxnStatusTablet state changed. Reason: New leader ffab48f314254d6999e81e2dd3aed8d4. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.174621 13791 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:56:34.174845 13824 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260621 07:56:34.175000 13824 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260621 07:56:34.175468 13274 catalog_manager.cc:5495] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131). New cstate: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } health_report { overall_health: HEALTHY } } }
I20260621 07:56:34.191416 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.188068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36767 (local address 127.0.0.1:35928)
0621 07:56:34.188204 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:34.188227 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:34.188307 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:34.188626 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:34.188629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:34.188635 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:34.188718 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.188722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.189670 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.189673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.190265 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.190273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.190804 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.190806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.190864 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.190903 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.191180 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.191190 (+ 10us) client_negotiation.cc:769] Sending connection context
0621 07:56:34.191239 (+ 49us) client_negotiation.cc:240] Negotiation successful
0621 07:56:34.191312 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260621 07:56:34.191437 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.188130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.11.223.131:36767)
0621 07:56:34.188237 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.188242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.188257 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:56:34.188474 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.188477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.188524 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.188604 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:34.188810 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.189562 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.190364 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.190687 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.190802 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.191016 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.191090 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.191163 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.191249 (+ 86us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.191315 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:56:34.193559 13428 raft_consensus.cc:1261] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Refusing update from remote peer ffab48f314254d6999e81e2dd3aed8d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:34.193595 13571 raft_consensus.cc:1261] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Refusing update from remote peer ffab48f314254d6999e81e2dd3aed8d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:34.193935 13791 consensus_queue.cc:1035] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:34.194226 13791 consensus_queue.cc:1035] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:34.195454 13786 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: TxnStatusTablet state changed. Reason: New leader ffab48f314254d6999e81e2dd3aed8d4. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.195511 13787 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: TxnStatusTablet state changed. Reason: New leader ffab48f314254d6999e81e2dd3aed8d4. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.195569 13786 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.195595 13787 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.197045 13791 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.197141 13791 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:56:34.197760 13788 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.197841 13788 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ffab48f314254d6999e81e2dd3aed8d4: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:56:34.198516 13787 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.198611 13787 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P ab2eea207f6a40f3b36c78914c269e7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.200078 13786 tablet_replica.cc:439] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffab48f314254d6999e81e2dd3aed8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab2eea207f6a40f3b36c78914c269e7d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 40677 } } peers { permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 } } peers { permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 } } }
I20260621 07:56:34.200181 13786 tablet_replica.cc:442] T 9714f9783bc940db8ea399b84e7178d6 P 8204082c65e14f4998a9bc885ca9806d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:34.206527 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.203043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.11.223.129:40677)
0621 07:56:34.203135 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.203138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.203162 (+ 24us) server_negotiation.cc:407] Connection header received
0621 07:56:34.203270 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.203273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.203325 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.203420 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:34.203572 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.204284 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.205038 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.205422 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.205514 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.205713 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.206198 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.206307 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.206360 (+ 53us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.206412 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260621 07:56:34.216363 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.213109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:33697 (local address 127.11.223.131:36767)
0621 07:56:34.213225 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.213228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.213435 (+ 207us) server_negotiation.cc:407] Connection header received
0621 07:56:34.213589 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.213593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.213643 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.213720 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:34.213957 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.214422 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.215254 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.215488 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.215595 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.215882 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.215954 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.216033 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.216152 (+ 119us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.216200 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:56:34.216375 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.213026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36767 (local address 127.11.223.129:33697)
0621 07:56:34.213357 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:34.213368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:34.213493 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:34.213732 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:34.213734 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:34.213741 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:34.213853 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.213858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.214545 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.214549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.215122 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.215147 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.215596 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.215599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.215692 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.215746 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.216039 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.216047 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:34.216130 (+ 83us) client_negotiation.cc:240] Negotiation successful
0621 07:56:34.216200 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260621 07:56:34.222450 13571 raft_consensus.cc:1261] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Refusing update from remote peer ab2eea207f6a40f3b36c78914c269e7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:34.222661 13711 raft_consensus.cc:1261] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Refusing update from remote peer ab2eea207f6a40f3b36c78914c269e7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:34.223201 13811 consensus_queue.cc:1035] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8204082c65e14f4998a9bc885ca9806d" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:34.223364 13787 consensus_queue.cc:1035] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffab48f314254d6999e81e2dd3aed8d4" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:34.244019 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.240464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:51871 (local address 127.11.223.129:40677)
0621 07:56:34.240554 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.240558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.240752 (+ 194us) server_negotiation.cc:407] Connection header received
0621 07:56:34.240884 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.240887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.240923 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.241004 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:34.241262 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.241925 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.242994 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.243224 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.243342 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.243594 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.243670 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.243748 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.243845 (+ 97us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.243888 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260621 07:56:34.244050 13837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.240421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:40677 (local address 127.11.223.131:51871)
0621 07:56:34.240675 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:34.240686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:34.240796 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:34.241013 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:34.241016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:34.241023 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:34.241154 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.241159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.242053 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.242056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.242881 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:34.242890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.243338 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.243341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:34.243423 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.243470 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:34.243756 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:34.243765 (+ 9us) client_negotiation.cc:769] Sending connection context
0621 07:56:34.243836 (+ 71us) client_negotiation.cc:240] Negotiation successful
0621 07:56:34.243905 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260621 07:56:34.268285 13273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:34.268452 13273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:34.270665 13273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db651c9b4cf6444ebfe4cd428e381f19: Sending DeleteTablet for 3 replicas of tablet 1562a4292b604d55b912d19657f3e523
I20260621 07:56:34.271281 13551 tablet_service.cc:1514] Processing DeleteTablet for tablet 1562a4292b604d55b912d19657f3e523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:34 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:32782
I20260621 07:56:34.271384 13406 tablet_service.cc:1514] Processing DeleteTablet for tablet 1562a4292b604d55b912d19657f3e523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:34 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:56366
I20260621 07:56:34.271435 13803 tablet_replica.cc:330] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: stopping tablet replica
I20260621 07:56:34.271528 13803 raft_consensus.cc:2229] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:34.271536 13804 tablet_replica.cc:330] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: stopping tablet replica
I20260621 07:56:34.271551 13691 tablet_service.cc:1514] Processing DeleteTablet for tablet 1562a4292b604d55b912d19657f3e523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:34 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35914
I20260621 07:56:34.271620 13803 raft_consensus.cc:2258] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:34.271624 13804 raft_consensus.cc:2229] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:56:34.271854 13804 raft_consensus.cc:2258] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:34.272023 13805 tablet_replica.cc:330] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: stopping tablet replica
I20260621 07:56:34.272127 13805 raft_consensus.cc:2229] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:34.272223 13805 raft_consensus.cc:2258] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:34.272642 13804 ts_tablet_manager.cc:1905] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:34.272869 13805 ts_tablet_manager.cc:1905] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:34.273551 13803 ts_tablet_manager.cc:1905] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:34.275336 13804 ts_tablet_manager.cc:1918] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:34.275408 13804 log.cc:1196] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/1562a4292b604d55b912d19657f3e523
I20260621 07:56:34.275535 13805 ts_tablet_manager.cc:1918] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:34.275604 13805 log.cc:1196] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/1562a4292b604d55b912d19657f3e523
I20260621 07:56:34.275655 13804 ts_tablet_manager.cc:1939] T 1562a4292b604d55b912d19657f3e523 P ab2eea207f6a40f3b36c78914c269e7d: Deleting consensus metadata
I20260621 07:56:34.275843 13805 ts_tablet_manager.cc:1939] T 1562a4292b604d55b912d19657f3e523 P ffab48f314254d6999e81e2dd3aed8d4: Deleting consensus metadata
I20260621 07:56:34.276268 13261 catalog_manager.cc:4841] TS ab2eea207f6a40f3b36c78914c269e7d (127.11.223.129:40677): tablet 1562a4292b604d55b912d19657f3e523 (table test-table [id=8275c6e05667433eaefae47452099b1c]) successfully deleted
I20260621 07:56:34.276476 13261 catalog_manager.cc:4841] TS ffab48f314254d6999e81e2dd3aed8d4 (127.11.223.131:36767): tablet 1562a4292b604d55b912d19657f3e523 (table test-table [id=8275c6e05667433eaefae47452099b1c]) successfully deleted
I20260621 07:56:34.276523 13803 ts_tablet_manager.cc:1918] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:34.276589 13803 log.cc:1196] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/1562a4292b604d55b912d19657f3e523
I20260621 07:56:34.276855 13803 ts_tablet_manager.cc:1939] T 1562a4292b604d55b912d19657f3e523 P 8204082c65e14f4998a9bc885ca9806d: Deleting consensus metadata
I20260621 07:56:34.277467 13260 catalog_manager.cc:4841] TS 8204082c65e14f4998a9bc885ca9806d (127.11.223.130:43435): tablet 1562a4292b604d55b912d19657f3e523 (table test-table [id=8275c6e05667433eaefae47452099b1c]) successfully deleted
Jun 21 07:56:34 dist-test-slave-0pn3 krb5kdc[13229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:34.289345 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.277952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.11.223.129:40677)
0621 07:56:34.278072 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.278077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.278505 (+ 428us) server_negotiation.cc:407] Connection header received
0621 07:56:34.278596 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.278599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.278644 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.278726 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:34.279544 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.280353 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.281553 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.281818 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.281927 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.285480 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:34.285488 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:34.285501 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:34.285525 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:34.287478 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:34.287940 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:34.287945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:34.287947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:34.288042 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:34.288383 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:34.288386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:34.288388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:34.288650 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:34.288844 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.289054 (+ 210us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.289165 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260621 07:56:34.289690 13470 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56388
I20260621 07:56:34.299556 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:34.291234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.11.223.190:43689)
0621 07:56:34.291374 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:34.291378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:34.291741 (+ 363us) server_negotiation.cc:407] Connection header received
0621 07:56:34.291830 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:34.291832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:34.291877 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:34.291956 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:34.292697 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.293492 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.294559 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:34.294883 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:34.294973 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:34.296307 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:34.296362 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:34.296365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:34.296392 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:34.297882 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:34.298354 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:34.298357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:34.298359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:34.298444 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:34.298725 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:34.298728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:34.298730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:34.298926 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:34.299049 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:34.299309 (+ 260us) server_negotiation.cc:299] Negotiation successful
0621 07:56:34.299417 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260621 07:56:34.299832 13273 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45244
I20260621 07:56:34.301035 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13346
I20260621 07:56:34.308333 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13486
I20260621 07:56:34.314196 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13626
I20260621 07:56:34.320536 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13242
2026-06-21T07:56:34Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2749 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:34 dist-test-slave-0pn3 krb5kdc[13847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:34 dist-test-slave-0pn3 krb5kdc[13847](info): set up 2 sockets
Jun 21 07:56:34 dist-test-slave-0pn3 krb5kdc[13847](info): commencing operation
krb5kdc: starting...
W20260621 07:56:35.320629 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:35 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:35.432575 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:39461
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:35.535753 13860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:35.536011 13860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:35.536077 13860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:35.539749 13860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:35.539821 13860 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260621 07:56:35.539851 13860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:35.539887 13860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:35.539918 13860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:35.539949 13860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:35.544759 13860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:39461
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:39461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:35.545687 13860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:35.546624 13860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:35.555455 13866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:35.555446 13868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:35.555446 13865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:35.555588 13860 server_base.cc:1034] running on GCE node
I20260621 07:56:35.556383 13860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:35.557425 13860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:35.558578 13860 hybrid_clock.cc:648] HybridClock initialized: now 1782028595558542 us; error 47 us; skew 500 ppm
Jun 21 07:56:35 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028595, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:35.561236 13860 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:35.561980 13860 webserver.cc:460] Webserver started at http://127.11.223.190:45833/ using document root <none> and password file <none>
I20260621 07:56:35.562445 13860 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:35.562497 13860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:35.562644 13860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:35.564528 13860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "948a8909d01e4989b12ebafc8e9974d7"
format_stamp: "Formatted at 2026-06-21 07:56:35 on dist-test-slave-0pn3"
server_key: "e21c9a950013c53eb2a758ec8c38ef3f"
server_key_iv: "a4069bae6311b6f96ef2c099395c2528"
server_key_version: "encryptionkey@0"
I20260621 07:56:35.565014 13860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "948a8909d01e4989b12ebafc8e9974d7"
format_stamp: "Formatted at 2026-06-21 07:56:35 on dist-test-slave-0pn3"
server_key: "e21c9a950013c53eb2a758ec8c38ef3f"
server_key_iv: "a4069bae6311b6f96ef2c099395c2528"
server_key_version: "encryptionkey@0"
I20260621 07:56:35.568246 13860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:35.570504 13875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:35.571616 13860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:35.571744 13860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "948a8909d01e4989b12ebafc8e9974d7"
format_stamp: "Formatted at 2026-06-21 07:56:35 on dist-test-slave-0pn3"
server_key: "e21c9a950013c53eb2a758ec8c38ef3f"
server_key_iv: "a4069bae6311b6f96ef2c099395c2528"
server_key_version: "encryptionkey@0"
I20260621 07:56:35.571838 13860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:35.588882 13860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:35.591393 13860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:35.591600 13860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:35.598888 13860 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:39461
I20260621 07:56:35.598920 13937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:39461 every 8 connection(s)
I20260621 07:56:35.599740 13860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:35.602265 13938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:35.607219 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Bootstrap starting.
I20260621 07:56:35.607642 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13860
I20260621 07:56:35.607748 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:35.608007 12158 external_mini_cluster.cc:1428] Setting key c836b0bf2a39ef14988d72c6a612c515
I20260621 07:56:35.609867 13938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:35.610549 13938 log.cc:824] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:35.612653 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: No bootstrap required, opened a new log
Jun 21 07:56:35 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:35.616930 13938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "948a8909d01e4989b12ebafc8e9974d7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 39461 } }
I20260621 07:56:35.617187 13938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:35.617261 13938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948a8909d01e4989b12ebafc8e9974d7, State: Initialized, Role: FOLLOWER
I20260621 07:56:35.617678 13938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "948a8909d01e4989b12ebafc8e9974d7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 39461 } }
I20260621 07:56:35.617826 13938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:35.617906 13938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:35.618017 13938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:35.618804 13938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "948a8909d01e4989b12ebafc8e9974d7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 39461 } }
I20260621 07:56:35.619120 13938 leader_election.cc:302] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 948a8909d01e4989b12ebafc8e9974d7; no voters:
I20260621 07:56:35.619442 13938 leader_election.cc:288] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:35.619516 13943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:35.619717 13943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [term 1 LEADER]: Becoming Leader. State: Replica: 948a8909d01e4989b12ebafc8e9974d7, State: Running, Role: LEADER
I20260621 07:56:35.620047 13943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "948a8909d01e4989b12ebafc8e9974d7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 39461 } }
I20260621 07:56:35.620671 13938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:35.622570 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:35.609301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.11.223.190:39461)
0621 07:56:35.609744 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:35.609758 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:35.609791 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:56:35.610439 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:35.610454 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:35.610810 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:35.611167 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:35.611955 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.613021 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.614248 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.614663 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.614780 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:35.617508 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:35.617513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:35.617524 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:35.617547 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:35.619381 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:35.619911 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:35.619917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:35.619922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:35.620055 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:35.620378 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:35.620381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:35.620383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:35.620785 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:35.620939 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:35.621337 (+ 398us) server_negotiation.cc:299] Negotiation successful
0621 07:56:35.621655 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":136,"threads_started":1}
I20260621 07:56:35.623021 13945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "948a8909d01e4989b12ebafc8e9974d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "948a8909d01e4989b12ebafc8e9974d7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 39461 } } }
I20260621 07:56:35.623257 13945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:35.621865 13944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 948a8909d01e4989b12ebafc8e9974d7. Latest consensus state: current_term: 1 leader_uuid: "948a8909d01e4989b12ebafc8e9974d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "948a8909d01e4989b12ebafc8e9974d7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 39461 } } }
I20260621 07:56:35.623471 13944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7 [sys.catalog]: This master's current role is: LEADER
W20260621 07:56:35.625145 13957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:56:35.625240 13957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:56:35.625324 13959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:35.626065 13959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:35.631174 13959 catalog_manager.cc:1347] Generated new cluster ID: 0e5f8d0d098b4e0b83485b0737a87e50
I20260621 07:56:35.631250 13959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:35.651597 13959 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:35.652447 13959 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:35.659766 13959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Generated new TSK 0
I20260621 07:56:35.660435 13959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:35.689323 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:39461
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:35.792785 13964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:35.793008 13964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:35.793048 13964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:35.796953 13964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:35.796999 13964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:35.797035 13964 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260621 07:56:35.797048 13964 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:56:35.797127 13964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:35.802027 13964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:35.802879 13964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:35.803820 13964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:35.812628 13970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:35.812747 13964 server_base.cc:1034] running on GCE node
W20260621 07:56:35.812588 13969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:35.812588 13972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:35.813349 13964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:35.813907 13964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:35.815081 13964 hybrid_clock.cc:648] HybridClock initialized: now 1782028595815065 us; error 27 us; skew 500 ppm
Jun 21 07:56:35 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028595, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:35.817777 13964 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:35.818524 13964 webserver.cc:460] Webserver started at http://127.11.223.129:37093/ using document root <none> and password file <none>
I20260621 07:56:35.819001 13964 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:35.819056 13964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:35.819204 13964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:35.821208 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "9d60398cd64341749f57d78e115d2590"
format_stamp: "Formatted at 2026-06-21 07:56:35 on dist-test-slave-0pn3"
server_key: "a06e8dab226590d235f9c85a6a1283b5"
server_key_iv: "03829357917ce5aa3b7b3ba886a7d578"
server_key_version: "encryptionkey@0"
I20260621 07:56:35.821723 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "9d60398cd64341749f57d78e115d2590"
format_stamp: "Formatted at 2026-06-21 07:56:35 on dist-test-slave-0pn3"
server_key: "a06e8dab226590d235f9c85a6a1283b5"
server_key_iv: "03829357917ce5aa3b7b3ba886a7d578"
server_key_version: "encryptionkey@0"
I20260621 07:56:35.825050 13964 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:35.827075 13979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:35.828191 13964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:35.828277 13964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "9d60398cd64341749f57d78e115d2590"
format_stamp: "Formatted at 2026-06-21 07:56:35 on dist-test-slave-0pn3"
server_key: "a06e8dab226590d235f9c85a6a1283b5"
server_key_iv: "03829357917ce5aa3b7b3ba886a7d578"
server_key_version: "encryptionkey@0"
I20260621 07:56:35.828423 13964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:35.838043 13964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:35.841064 13964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:35.841197 13964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:35.842471 13964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:35.842519 13964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:35.842585 13964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:35.842631 13964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:56:35 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028595, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:35.853412 13964 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:46071
I20260621 07:56:35.853507 14094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:46071 every 8 connection(s)
I20260621 07:56:35.854493 13964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:35.856000 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:35.843242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:40063 (local address 127.11.223.190:39461)
0621 07:56:35.843368 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:35.843373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:35.844175 (+ 802us) server_negotiation.cc:407] Connection header received
0621 07:56:35.845265 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:35.845268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:35.845328 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:35.845390 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:35.847033 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.847908 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.849282 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.849721 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.849847 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:35.852449 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:35.852452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:35.852454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:35.852476 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:35.854011 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:35.854591 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:35.854600 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:35.854602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:35.854689 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:35.855006 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:35.855009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:35.855011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:35.855212 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:35.855309 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:35.855787 (+ 478us) server_negotiation.cc:299] Negotiation successful
0621 07:56:35.855888 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260621 07:56:35.856709 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:35.843598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.11.223.129:40063)
0621 07:56:35.844019 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:35.844065 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:35.844994 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:35.845833 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:35.845841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:35.846339 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:35.846897 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:35.846920 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.848056 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.848061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:35.849173 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:35.849184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.849912 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.849916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:35.850151 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:35.850680 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:35.850701 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:35.852236 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:35.854128 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:35.854132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:35.854142 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:35.854462 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:35.854778 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:35.854781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:35.854782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:35.854899 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:35.855357 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:35.855362 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:35.855559 (+ 197us) client_negotiation.cc:769] Sending connection context
0621 07:56:35.855701 (+ 142us) client_negotiation.cc:240] Negotiation successful
0621 07:56:35.855910 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
I20260621 07:56:35.857889 14095 heartbeater.cc:347] Connected to a master server at 127.11.223.190:39461
I20260621 07:56:35.858032 14095 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:35.858495 14095 heartbeater.cc:511] Master 127.11.223.190:39461 requested a full tablet report, sending...
I20260621 07:56:35.860121 13891 ts_manager.cc:194] Registered new tserver with Master: 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071)
I20260621 07:56:35.861231 13891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:40063
I20260621 07:56:35.864550 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 13964
I20260621 07:56:35.864683 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:35.864938 12158 external_mini_cluster.cc:1428] Setting key 8a44a781084fbaf81fd3e2704038a99f
I20260621 07:56:35.866637 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:35.860011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.11.223.129:54179)
0621 07:56:35.860358 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:35.860370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:35.860480 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:35.860692 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:35.860694 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:35.860883 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:35.860965 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:35.860970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.861743 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.861746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:35.862546 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:35.862556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.862981 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.862984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:35.863066 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:35.863446 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:35.863457 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:35.863731 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:35.865388 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:35.865393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:35.865397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:35.865581 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:35.865836 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:35.865838 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:35.865839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:35.865907 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:35.866239 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:35.866242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:35.866335 (+ 93us) client_negotiation.cc:769] Sending connection context
0621 07:56:35.866420 (+ 85us) client_negotiation.cc:240] Negotiation successful
0621 07:56:35.866509 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":94,"threads_started":1}
I20260621 07:56:35.866748 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:35.860080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:54179 (local address 127.11.223.190:39461)
0621 07:56:35.860199 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:35.860203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:35.860444 (+ 241us) server_negotiation.cc:407] Connection header received
0621 07:56:35.860551 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:35.860554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:35.860592 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:35.860679 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:35.861071 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.861619 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.862686 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:35.862878 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:35.862948 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:35.863841 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:35.863844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:35.863845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:35.863864 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:35.865291 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:35.865666 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:35.865669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:35.865671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:35.865753 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:35.865984 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:35.865987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:35.865989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:35.866156 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:35.866229 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:35.866501 (+ 272us) server_negotiation.cc:299] Negotiation successful
0621 07:56:35.866600 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:35.889322 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:39461
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:35.992877 14104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:35.993101 14104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:35.993162 14104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:35.996990 14104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:35.997035 14104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:35.997072 14104 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260621 07:56:35.997085 14104 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:56:35.997114 14104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:36.002087 14104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:36.003043 14104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:36.003955 14104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:36.012744 14112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:36.012773 14110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:36.012786 14104 server_base.cc:1034] running on GCE node
W20260621 07:56:36.012744 14109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:36.013348 14104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:36.013914 14104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:36.015092 14104 hybrid_clock.cc:648] HybridClock initialized: now 1782028596015081 us; error 26 us; skew 500 ppm
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028596, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:36.017709 14104 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:36.018527 14104 webserver.cc:460] Webserver started at http://127.11.223.130:41033/ using document root <none> and password file <none>
I20260621 07:56:36.019071 14104 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:36.019161 14104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:36.019402 14104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:36.021552 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "35be65c2d2b5417b847bfb3d635afc63"
format_stamp: "Formatted at 2026-06-21 07:56:36 on dist-test-slave-0pn3"
server_key: "3391f4523888d9a7870f1bd984b9502d"
server_key_iv: "a89652a8baeb215551c506c731180982"
server_key_version: "encryptionkey@0"
I20260621 07:56:36.022195 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "35be65c2d2b5417b847bfb3d635afc63"
format_stamp: "Formatted at 2026-06-21 07:56:36 on dist-test-slave-0pn3"
server_key: "3391f4523888d9a7870f1bd984b9502d"
server_key_iv: "a89652a8baeb215551c506c731180982"
server_key_version: "encryptionkey@0"
I20260621 07:56:36.025652 14104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:36.027842 14119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:36.029067 14104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:36.029197 14104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "35be65c2d2b5417b847bfb3d635afc63"
format_stamp: "Formatted at 2026-06-21 07:56:36 on dist-test-slave-0pn3"
server_key: "3391f4523888d9a7870f1bd984b9502d"
server_key_iv: "a89652a8baeb215551c506c731180982"
server_key_version: "encryptionkey@0"
I20260621 07:56:36.029304 14104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:36.042799 14104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:36.045463 14104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:36.045651 14104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:36.046936 14104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:36.047008 14104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:36.047077 14104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:36.047120 14104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028596, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:36.057803 14104 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:46667
I20260621 07:56:36.058599 14234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:46667 every 8 connection(s)
I20260621 07:56:36.058714 14104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:36.060504 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.047749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:44859 (local address 127.11.223.190:39461)
0621 07:56:36.047866 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.047870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.048892 (+ 1022us) server_negotiation.cc:407] Connection header received
0621 07:56:36.049775 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.049778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.049838 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.049916 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:36.051414 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.052307 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.053591 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.053835 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.053947 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.056737 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:36.056740 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:36.056743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:36.056767 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:36.058421 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.058960 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.058962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.058964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.059038 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.059496 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.059499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.059502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.059714 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:36.059809 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.060238 (+ 429us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.060379 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260621 07:56:36.061195 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.048270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.11.223.130:44859)
0621 07:56:36.048744 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.048782 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.049572 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.050114 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.050140 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.050623 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:36.051259 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.051282 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.052431 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.052434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.053460 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.053469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.053994 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.053998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.054166 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.054736 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:36.054754 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:36.056572 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:36.058523 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.058528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.058541 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.058868 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.059167 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.059172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.059174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.059370 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.059813 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:36.059817 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:36.060027 (+ 210us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.060167 (+ 140us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.060458 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":121,"threads_started":1}
I20260621 07:56:36.062307 14235 heartbeater.cc:347] Connected to a master server at 127.11.223.190:39461
I20260621 07:56:36.062429 14235 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:36.062932 14235 heartbeater.cc:511] Master 127.11.223.190:39461 requested a full tablet report, sending...
I20260621 07:56:36.063910 13891 ts_manager.cc:194] Registered new tserver with Master: 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667)
I20260621 07:56:36.064647 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 14104
I20260621 07:56:36.064683 13891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:44859
I20260621 07:56:36.064769 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:36.065136 12158 external_mini_cluster.cc:1428] Setting key 19bbde7812a2f38dad2531f3ae937a07
I20260621 07:56:36.073048 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.064437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:39749 (local address 127.11.223.190:39461)
0621 07:56:36.064583 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.064587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.065751 (+ 1164us) server_negotiation.cc:407] Connection header received
0621 07:56:36.065838 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.065841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.065886 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.065966 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:36.066472 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.067022 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.068291 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.068711 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.068828 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.069976 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:36.069979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:36.069981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:36.070000 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:36.071425 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.071890 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.071892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.071893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.071956 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.072235 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.072237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.072238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.072482 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:36.072556 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.072866 (+ 310us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.072947 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260621 07:56:36.073184 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.064344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.11.223.130:39749)
0621 07:56:36.065656 (+ 1312us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.065671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.065759 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.065998 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.066001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.066198 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:36.066351 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.066359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.067155 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.067162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.068165 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.068179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.068820 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.068823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.068890 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.069385 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:36.069404 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:36.069842 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:36.071531 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.071535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.071537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.071799 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.072039 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.072044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.072047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.072145 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.072575 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:36.072580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:36.072693 (+ 113us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.072822 (+ 129us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.072960 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:36.091049 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:39461
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:36.194938 14244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:36.195209 14244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:36.195281 14244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:36.199393 14244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:36.199484 14244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:36.199550 14244 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260621 07:56:36.199589 14244 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:56:36.199646 14244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:36.204730 14244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:36.205642 14244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:36.206564 14244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:36.215323 14252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:36.215345 14250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:36.215466 14244 server_base.cc:1034] running on GCE node
W20260621 07:56:36.215341 14249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:36.215993 14244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:36.216629 14244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:36.217803 14244 hybrid_clock.cc:648] HybridClock initialized: now 1782028596217778 us; error 37 us; skew 500 ppm
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028596, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:36.220485 14244 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:36.221386 14244 webserver.cc:460] Webserver started at http://127.11.223.131:33057/ using document root <none> and password file <none>
I20260621 07:56:36.221925 14244 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:36.222011 14244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:36.222250 14244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:36.224152 14244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "30f055617ecd407ea9902180927f81e3"
format_stamp: "Formatted at 2026-06-21 07:56:36 on dist-test-slave-0pn3"
server_key: "eeb3c01490168e5dea1562679525e271"
server_key_iv: "e856370d7952676758b75a47df326d60"
server_key_version: "encryptionkey@0"
I20260621 07:56:36.224722 14244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "30f055617ecd407ea9902180927f81e3"
format_stamp: "Formatted at 2026-06-21 07:56:36 on dist-test-slave-0pn3"
server_key: "eeb3c01490168e5dea1562679525e271"
server_key_iv: "e856370d7952676758b75a47df326d60"
server_key_version: "encryptionkey@0"
I20260621 07:56:36.228080 14244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:36.230376 14259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:36.231516 14244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:36.231648 14244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "30f055617ecd407ea9902180927f81e3"
format_stamp: "Formatted at 2026-06-21 07:56:36 on dist-test-slave-0pn3"
server_key: "eeb3c01490168e5dea1562679525e271"
server_key_iv: "e856370d7952676758b75a47df326d60"
server_key_version: "encryptionkey@0"
I20260621 07:56:36.231751 14244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:36.252715 14244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:36.255707 14244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:36.255892 14244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:36.257284 14244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:36.257352 14244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:36.257421 14244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:36.257474 14244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028596, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:36.268369 14244 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:38085
I20260621 07:56:36.269236 14374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:38085 every 8 connection(s)
I20260621 07:56:36.269469 14244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:36.271031 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.257958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:57603 (local address 127.11.223.190:39461)
0621 07:56:36.258081 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.258084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.259597 (+ 1513us) server_negotiation.cc:407] Connection header received
0621 07:56:36.260877 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.260883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.260954 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.261058 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:36.262344 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.263212 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.264696 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.264925 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.265031 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.267378 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:36.267382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:36.267384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:36.267412 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:36.269060 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.269605 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.269609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.269610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.269690 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.270042 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.270045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.270047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.270299 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:36.270402 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.270827 (+ 425us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.270921 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:56:36.271732 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.258975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.11.223.131:57603)
0621 07:56:36.259421 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.259487 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.260641 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.261231 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.261238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.261623 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:36.262190 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.262206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.263338 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.263342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.264561 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.264570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.265064 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.265068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.265235 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.265753 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:36.265767 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:36.267243 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:36.269182 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.269186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.269202 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.269519 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.269782 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.269786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.269788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.269939 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.270401 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:36.270405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:36.270612 (+ 207us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.270767 (+ 155us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.270977 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260621 07:56:36.272858 14375 heartbeater.cc:347] Connected to a master server at 127.11.223.190:39461
I20260621 07:56:36.272992 14375 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:36.273478 14375 heartbeater.cc:511] Master 127.11.223.190:39461 requested a full tablet report, sending...
I20260621 07:56:36.274528 13891 ts_manager.cc:194] Registered new tserver with Master: 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.275242 13891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:57603
I20260621 07:56:36.276392 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 14244
I20260621 07:56:36.276490 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:36.276840 12158 external_mini_cluster.cc:1428] Setting key c499ea3eba3ca477c03f484dbf0fc85b
I20260621 07:56:36.278764 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:56:36.282359 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.275032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:45093 (local address 127.11.223.190:39461)
0621 07:56:36.275137 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.275141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.275272 (+ 131us) server_negotiation.cc:407] Connection header received
0621 07:56:36.275385 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.275388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.275432 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.275513 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:36.275973 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.276625 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.277681 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.277893 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.277994 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.279232 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:36.279235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:36.279238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:36.279261 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:36.280811 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.281240 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.281245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.281247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.281328 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.281619 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.281622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.281624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.281789 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:36.281890 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.282126 (+ 236us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.282218 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:56:36.282371 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.274917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.11.223.131:45093)
0621 07:56:36.275186 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.275200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.275294 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.275546 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.275548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.275743 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:36.275865 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.275872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.276801 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.276804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.277558 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.277567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.277998 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.278001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.278088 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.278749 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:36.278760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:36.279097 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:36.280938 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.280941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.280943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.281140 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.281441 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.281443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.281445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.281527 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.281881 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:36.281884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:36.281968 (+ 84us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.282076 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.282212 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:36.302280 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.295301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.11.223.190:39461)
0621 07:56:36.295411 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.295414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.295425 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:36.295453 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.295455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.295486 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.295547 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:36.296169 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.296683 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.297416 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.297601 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.297670 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.299778 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:36.299781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:36.299783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:36.299801 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:36.301051 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.301415 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.301417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.301419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.301484 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.301698 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.301700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.301701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.301838 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:36.301931 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.302095 (+ 164us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.302170 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:56:36.304908 13891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:36.307761 13891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:36.321292 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.315821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.11.223.131:38085)
0621 07:56:36.316094 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.316105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.316476 (+ 371us) server_negotiation.cc:407] Connection header received
0621 07:56:36.316708 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.316714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.316920 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.317036 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.317410 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.318476 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.319863 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.320609 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.320742 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.320849 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.321056 (+ 207us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.321146 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260621 07:56:36.321292 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.315397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.0.0.1:35346)
0621 07:56:36.316166 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.316214 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.316349 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.316920 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.316926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.316949 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.317075 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.317081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.318168 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.318170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.319515 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.319524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.320800 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.320803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.320920 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.320937 (+ 17us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.321035 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.321147 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":90,"thread_start_us":87,"threads_started":1}
I20260621 07:56:36.321311 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.315660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38085 (local address 127.0.0.1:50638)
0621 07:56:36.316357 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.316373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.316547 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.317192 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.317194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.317210 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.317292 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.317297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.318595 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.318598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.319741 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.319749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.320726 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.320730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.320862 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.320901 (+ 39us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.321011 (+ 110us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.321145 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":61,"thread_start_us":90,"threads_started":1}
I20260621 07:56:36.321684 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.316103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:46667 (local address 127.0.0.1:51394)
0621 07:56:36.317195 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.317205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.317270 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.317923 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.317927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.317944 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.318034 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.318039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.319114 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.319121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.320240 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.320253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.321241 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.321244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.321367 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.321391 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.321447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.321512 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":61,"thread_start_us":90,"threads_started":1}
I20260621 07:56:36.322211 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.316531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.11.223.130:46667)
0621 07:56:36.317515 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.317526 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.317546 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:56:36.317612 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.317616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.317786 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.317920 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.318151 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.319003 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.320400 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.321114 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.321938 (+ 824us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.322022 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.322057 (+ 35us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.322112 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":106,"threads_started":1}
I20260621 07:56:36.322703 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.315529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.11.223.129:46071)
0621 07:56:36.316232 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.316244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.316271 (+ 27us) server_negotiation.cc:407] Connection header received
0621 07:56:36.316541 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.316547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.316759 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.316917 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.317219 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.318055 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.319641 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.320469 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.320601 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.320737 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.322478 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.322564 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":120,"threads_started":1}
I20260621 07:56:36.324478 14169 tablet_service.cc:1467] Processing CreateTablet for tablet fe4772e3e0244a0ebdc0347b06dd1ca3 (DEFAULT_TABLE table=test-table [id=720e2e600459408487ec2a2b65119701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:36.324460 14309 tablet_service.cc:1467] Processing CreateTablet for tablet fe4772e3e0244a0ebdc0347b06dd1ca3 (DEFAULT_TABLE table=test-table [id=720e2e600459408487ec2a2b65119701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:36.325359 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe4772e3e0244a0ebdc0347b06dd1ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.325057 14029 tablet_service.cc:1467] Processing CreateTablet for tablet fe4772e3e0244a0ebdc0347b06dd1ca3 (DEFAULT_TABLE table=test-table [id=720e2e600459408487ec2a2b65119701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:36.325412 14309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe4772e3e0244a0ebdc0347b06dd1ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.325879 14029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe4772e3e0244a0ebdc0347b06dd1ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.331712 14398 tablet_bootstrap.cc:492] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Bootstrap starting.
I20260621 07:56:36.333027 14400 tablet_bootstrap.cc:492] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Bootstrap starting.
I20260621 07:56:36.333366 14399 tablet_bootstrap.cc:492] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Bootstrap starting.
I20260621 07:56:36.334187 14398 tablet_bootstrap.cc:654] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.335005 14398 log.cc:824] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:36.335124 14400 tablet_bootstrap.cc:654] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.335193 14399 tablet_bootstrap.cc:654] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.335882 14400 log.cc:824] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:36.335987 14399 log.cc:824] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:36.337357 14398 tablet_bootstrap.cc:492] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: No bootstrap required, opened a new log
I20260621 07:56:36.337551 14398 ts_tablet_manager.cc:1397] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 07:56:36.337705 14400 tablet_bootstrap.cc:492] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: No bootstrap required, opened a new log
I20260621 07:56:36.337705 14399 tablet_bootstrap.cc:492] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: No bootstrap required, opened a new log
I20260621 07:56:36.337877 14400 ts_tablet_manager.cc:1397] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 07:56:36.337877 14399 ts_tablet_manager.cc:1397] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 07:56:36.340435 14398 raft_consensus.cc:348] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.340658 14398 raft_consensus.cc:374] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.340735 14398 raft_consensus.cc:729] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.341142 14398 consensus_queue.cc:260] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.341715 14235 heartbeater.cc:503] Master 127.11.223.190:39461 was elected leader, sending a full tablet report...
I20260621 07:56:36.341609 14400 raft_consensus.cc:348] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.341609 14399 raft_consensus.cc:348] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.341845 14400 raft_consensus.cc:374] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.341879 14399 raft_consensus.cc:374] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.341892 14400 raft_consensus.cc:729] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.341924 14399 raft_consensus.cc:729] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f055617ecd407ea9902180927f81e3, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.342077 14398 ts_tablet_manager.cc:1428] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260621 07:56:36.342307 14400 consensus_queue.cc:260] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.342303 14399 consensus_queue.cc:260] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.342506 14404 raft_consensus.cc:482] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:36.342810 14404 raft_consensus.cc:504] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.343194 14399 ts_tablet_manager.cc:1428] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260621 07:56:36.343187 14400 ts_tablet_manager.cc:1428] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 07:56:36.343498 14375 heartbeater.cc:503] Master 127.11.223.190:39461 was elected leader, sending a full tablet report...
I20260621 07:56:36.344822 14095 heartbeater.cc:503] Master 127.11.223.190:39461 was elected leader, sending a full tablet report...
I20260621 07:56:36.348498 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.344108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.11.223.130:57685)
0621 07:56:36.344598 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.344613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.344713 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.345071 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.345074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.345089 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.345193 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.345199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.346099 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.346103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.347196 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.347208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.348007 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.348011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.348138 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.348180 (+ 42us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.348266 (+ 86us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.348373 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260621 07:56:36.348492 14404 leader_election.cc:288] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.348534 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.344433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:57685 (local address 127.11.223.129:46071)
0621 07:56:36.344537 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.344542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.344802 (+ 260us) server_negotiation.cc:407] Connection header received
0621 07:56:36.344848 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.344852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.344894 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.344969 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.345313 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.345974 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.347328 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.347910 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.348026 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.348077 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.348309 (+ 232us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.348419 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:56:36.349284 14049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe4772e3e0244a0ebdc0347b06dd1ca3" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d60398cd64341749f57d78e115d2590" is_pre_election: true
I20260621 07:56:36.349531 14049 raft_consensus.cc:2454] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 0.
I20260621 07:56:36.349920 14122 leader_election.cc:302] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35be65c2d2b5417b847bfb3d635afc63, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:36.350194 14405 raft_consensus.cc:2781] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:36.350296 14405 raft_consensus.cc:482] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:36.350375 14405 raft_consensus.cc:3037] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.351363 14405 raft_consensus.cc:504] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.351675 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.348075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:58245 (local address 127.11.223.131:38085)
0621 07:56:36.348181 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.348184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.348212 (+ 28us) server_negotiation.cc:407] Connection header received
0621 07:56:36.348285 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.348288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.348328 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.348416 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.348759 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.349332 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.350353 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.350805 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.350939 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.350991 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.351512 (+ 521us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.351577 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260621 07:56:36.351703 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.347762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38085 (local address 127.11.223.130:58245)
0621 07:56:36.348026 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.348039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.348116 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.348512 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.348515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.348531 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.348649 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.348657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.349447 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.349452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.350243 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.350252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.351322 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.351327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.351394 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.351411 (+ 17us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.351497 (+ 86us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.351560 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260621 07:56:36.351740 14405 leader_election.cc:288] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 election: Requested vote from peers 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.352013 14049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe4772e3e0244a0ebdc0347b06dd1ca3" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d60398cd64341749f57d78e115d2590"
I20260621 07:56:36.352108 14049 raft_consensus.cc:3037] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.352447 14328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe4772e3e0244a0ebdc0347b06dd1ca3" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3"
I20260621 07:56:36.352451 14329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe4772e3e0244a0ebdc0347b06dd1ca3" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3" is_pre_election: true
I20260621 07:56:36.352641 14328 raft_consensus.cc:3037] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.353363 14049 raft_consensus.cc:2454] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 1.
I20260621 07:56:36.353650 14122 leader_election.cc:302] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35be65c2d2b5417b847bfb3d635afc63, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:36.353814 14405 raft_consensus.cc:2781] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:36.353919 14405 raft_consensus.cc:686] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 LEADER]: Becoming Leader. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Running, Role: LEADER
I20260621 07:56:36.354272 14405 consensus_queue.cc:237] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.354297 14328 raft_consensus.cc:2454] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 1.
W20260621 07:56:36.355829 14096 tablet.cc:2369] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:36.357084 13891 catalog_manager.cc:5495] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "35be65c2d2b5417b847bfb3d635afc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } health_report { overall_health: UNKNOWN } } }
Jun 21 07:56:36 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028595, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:36.389117 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.381723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:39461 (local address 127.0.0.1:36604)
0621 07:56:36.382034 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.382045 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.382156 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.382408 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.382411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.382775 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:36.382859 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.382865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.383551 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.383558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.384093 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.384100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.384565 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.384569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.384642 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.385109 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:36.385120 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:36.386392 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:36.387740 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.387747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.387751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.387961 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.388189 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:36.388191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:36.388192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:36.388293 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:36.388667 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:36.388671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:36.388768 (+ 97us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.388857 (+ 89us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.388969 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":104,"threads_started":1}
I20260621 07:56:36.389137 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.381842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.11.223.190:39461)
0621 07:56:36.381975 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.381979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.382115 (+ 136us) server_negotiation.cc:407] Connection header received
0621 07:56:36.382269 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.382272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.382313 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.382392 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:36.382970 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.383452 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.384209 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.384463 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.384566 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.386509 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:36.386512 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:36.386514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:36.386534 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:36.387644 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.388041 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.388043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.388044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.388111 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:36.388419 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:36.388421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:36.388422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:36.388576 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:36.388651 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.388907 (+ 256us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.389020 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":59}
I20260621 07:56:36.392499 13891 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:36604:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260621 07:56:36.393186 13891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:36.397775 14309 tablet_service.cc:1467] Processing CreateTablet for tablet 745579ed2bb74b32a9bcec8dc656bb19 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c59a4c90e054d24a158444393f74159]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:56:36.397691 14029 tablet_service.cc:1467] Processing CreateTablet for tablet 745579ed2bb74b32a9bcec8dc656bb19 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c59a4c90e054d24a158444393f74159]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:56:36.397734 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 745579ed2bb74b32a9bcec8dc656bb19 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c59a4c90e054d24a158444393f74159]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:56:36.398433 14309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 745579ed2bb74b32a9bcec8dc656bb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.398433 14029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 745579ed2bb74b32a9bcec8dc656bb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.398445 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 745579ed2bb74b32a9bcec8dc656bb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.401010 14398 tablet_bootstrap.cc:492] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: Bootstrap starting.
I20260621 07:56:36.401162 14400 tablet_bootstrap.cc:492] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: Bootstrap starting.
I20260621 07:56:36.401309 14399 tablet_bootstrap.cc:492] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: Bootstrap starting.
I20260621 07:56:36.401952 14400 tablet_bootstrap.cc:654] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.401952 14398 tablet_bootstrap.cc:654] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.402012 14399 tablet_bootstrap.cc:654] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.403031 14399 tablet_bootstrap.cc:492] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: No bootstrap required, opened a new log
I20260621 07:56:36.403133 14399 ts_tablet_manager.cc:1397] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:36.403548 14399 raft_consensus.cc:348] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.403657 14399 raft_consensus.cc:374] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.403699 14399 raft_consensus.cc:729] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f055617ecd407ea9902180927f81e3, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.403841 14399 consensus_queue.cc:260] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.404102 14399 ts_tablet_manager.cc:1428] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260621 07:56:36.404095 14406 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.404307 14406 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.404976 14398 tablet_bootstrap.cc:492] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: No bootstrap required, opened a new log
I20260621 07:56:36.405048 14400 tablet_bootstrap.cc:492] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: No bootstrap required, opened a new log
I20260621 07:56:36.405061 14398 ts_tablet_manager.cc:1397] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260621 07:56:36.405123 14400 ts_tablet_manager.cc:1397] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260621 07:56:36.405488 14398 raft_consensus.cc:348] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.405526 14400 raft_consensus.cc:348] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.405587 14398 raft_consensus.cc:374] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.405614 14400 raft_consensus.cc:374] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.405617 14398 raft_consensus.cc:729] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.405646 14400 raft_consensus.cc:729] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.405751 14400 consensus_queue.cc:260] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.405745 14398 consensus_queue.cc:260] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.405997 14398 ts_tablet_manager.cc:1428] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:36.405970 14407 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.405989 14400 ts_tablet_manager.cc:1428] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:36.406123 14407 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.406834 14405 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.406948 14405 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.412866 14407 raft_consensus.cc:482] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:36.412961 14407 raft_consensus.cc:504] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.413864 14407 leader_election.cc:288] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.417757 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.414136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:46517 (local address 127.11.223.131:38085)
0621 07:56:36.414259 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.414264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.414402 (+ 138us) server_negotiation.cc:407] Connection header received
0621 07:56:36.414497 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.414499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.414530 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.414609 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.414965 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.415606 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.416852 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.417232 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.417361 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.417412 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.417583 (+ 171us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.417643 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:56:36.417783 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.414042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38085 (local address 127.11.223.129:46517)
0621 07:56:36.414331 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.414343 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.414420 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.414662 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.414665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.414712 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.414846 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.414851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.415719 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.415722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.416740 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.416748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.417361 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.417365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.417468 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.417485 (+ 17us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.417569 (+ 84us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.417635 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"spinlock_wait_cycles":4736,"thread_start_us":83,"threads_started":1}
I20260621 07:56:36.418123 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.414175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:53447 (local address 127.11.223.130:46667)
0621 07:56:36.414266 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.414271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.414679 (+ 408us) server_negotiation.cc:407] Connection header received
0621 07:56:36.414793 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.414795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.414827 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.414885 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:36.415538 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.416287 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.417210 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.417584 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.417712 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.417758 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.417948 (+ 190us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.418015 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260621 07:56:36.418109 14328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745579ed2bb74b32a9bcec8dc656bb19" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3" is_pre_election: true
I20260621 07:56:36.418247 14328 raft_consensus.cc:2454] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 0.
I20260621 07:56:36.418507 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.414042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:46667 (local address 127.11.223.129:53447)
0621 07:56:36.414574 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.414583 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.414705 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.414918 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.414923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.414945 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:36.415423 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.415439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.416417 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.416419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.417093 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.417101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.417695 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.417699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.417792 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.417818 (+ 26us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.418338 (+ 520us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.418405 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":20,"spinlock_wait_cycles":25856,"thread_start_us":65,"threads_started":1}
I20260621 07:56:36.418548 13981 leader_election.cc:302] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f055617ecd407ea9902180927f81e3, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:36.418835 14407 raft_consensus.cc:2781] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:36.418908 14407 raft_consensus.cc:482] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:36.418957 14407 raft_consensus.cc:3037] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.419066 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745579ed2bb74b32a9bcec8dc656bb19" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35be65c2d2b5417b847bfb3d635afc63" is_pre_election: true
I20260621 07:56:36.419277 14189 raft_consensus.cc:2454] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 0.
I20260621 07:56:36.419992 14407 raft_consensus.cc:504] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.420279 14407 leader_election.cc:288] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 election: Requested vote from peers 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.420604 14328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745579ed2bb74b32a9bcec8dc656bb19" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3"
I20260621 07:56:36.420735 14328 raft_consensus.cc:3037] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.420717 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745579ed2bb74b32a9bcec8dc656bb19" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35be65c2d2b5417b847bfb3d635afc63"
I20260621 07:56:36.420807 14189 raft_consensus.cc:3037] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.421770 14328 raft_consensus.cc:2454] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 1.
I20260621 07:56:36.421937 14189 raft_consensus.cc:2454] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 1.
I20260621 07:56:36.422026 13981 leader_election.cc:302] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f055617ecd407ea9902180927f81e3, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:36.422197 14407 raft_consensus.cc:2781] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:36.422407 14407 raft_consensus.cc:686] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [term 1 LEADER]: Becoming Leader. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Running, Role: LEADER
I20260621 07:56:36.422729 14407 consensus_queue.cc:237] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.424618 14421 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: TxnStatusTablet state changed. Reason: New leader 9d60398cd64341749f57d78e115d2590. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.424713 14421 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:56:36.424981 14423 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260621 07:56:36.425179 14423 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260621 07:56:36.425618 13890 catalog_manager.cc:5495] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d60398cd64341749f57d78e115d2590 (127.11.223.129). New cstate: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } health_report { overall_health: UNKNOWN } } }
I20260621 07:56:36.437330 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.433661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.11.223.129:46071)
0621 07:56:36.433780 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.433784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.433797 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:56:36.433858 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.433861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.433915 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.434014 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:36.434215 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.434933 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.435804 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.436125 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.436232 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.436548 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.436990 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.437103 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.437157 (+ 54us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.437215 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:56:36.437333 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.433411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.0.0.1:35354)
0621 07:56:36.433523 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.433535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.433642 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.434024 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.434027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.434033 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:36.434113 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.434118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.435039 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.435042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.435696 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.435705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.436259 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.436262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.436373 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.436421 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.437084 (+ 663us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.437094 (+ 10us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.437143 (+ 49us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.437208 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260621 07:56:36.441705 14328 raft_consensus.cc:1261] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Refusing update from remote peer 9d60398cd64341749f57d78e115d2590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:36.441847 14189 raft_consensus.cc:1261] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Refusing update from remote peer 9d60398cd64341749f57d78e115d2590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:36.442358 14407 consensus_queue.cc:1035] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:36.442531 14421 consensus_queue.cc:1035] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:36.445647 14405 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: TxnStatusTablet state changed. Reason: New leader 9d60398cd64341749f57d78e115d2590. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.445758 14405 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.445734 14406 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: TxnStatusTablet state changed. Reason: New leader 9d60398cd64341749f57d78e115d2590. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.445824 14406 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.449136 14407 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.449249 14407 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:56:36.450579 14406 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.450680 14406 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 30f055617ecd407ea9902180927f81e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.453193 14421 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.453300 14421 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 9d60398cd64341749f57d78e115d2590: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:56:36.454700 14405 tablet_replica.cc:439] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } } }
I20260621 07:56:36.454813 14405 tablet_replica.cc:442] T 745579ed2bb74b32a9bcec8dc656bb19 P 35be65c2d2b5417b847bfb3d635afc63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:56:36.454813 14428 mvcc.cc:205] Tried to move back new op lower bound from 7299189131019730944 to 7299189130949713920. Current Snapshot: MvccSnapshot[applied={T|T < 7299189131019730944}]
I20260621 07:56:36.459787 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.455984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.11.223.130:46667)
0621 07:56:36.456086 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.456090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.456102 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:56:36.456158 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.456161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.456212 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.456348 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:36.456590 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.457368 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.458297 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.458633 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.458735 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.458955 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.459414 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.459518 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.459576 (+ 58us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.459638 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260621 07:56:36.470129 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.466841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:33225 (local address 127.11.223.129:46071)
0621 07:56:36.466959 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.466963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.467013 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:56:36.467118 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.467120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.467154 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.467218 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:36.467430 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.467957 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.469171 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.469376 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.469457 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.469756 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.469812 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.469861 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.469972 (+ 111us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.470010 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:56:36.470175 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.466779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.11.223.130:33225)
0621 07:56:36.466960 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.466973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.467052 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.467240 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.467242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.467248 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:36.467343 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.467348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.468072 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.468077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.469049 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.469060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.469481 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.469485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.469579 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.469623 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.469887 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.469893 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.469956 (+ 63us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.470024 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260621 07:56:36.476558 14328 raft_consensus.cc:1261] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Refusing update from remote peer 35be65c2d2b5417b847bfb3d635afc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:36.476828 14049 raft_consensus.cc:1261] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Refusing update from remote peer 35be65c2d2b5417b847bfb3d635afc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:36.476917 14404 consensus_queue.cc:1035] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:36.477275 14404 consensus_queue.cc:1035] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:36.481385 14426 mvcc.cc:205] Tried to move back new op lower bound from 7299189131164016640 to 7299189130669895680. Current Snapshot: MvccSnapshot[applied={T|T < 7299189131164016640}]
I20260621 07:56:36.498641 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.494844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:46667 (local address 127.11.223.129:43139)
0621 07:56:36.495234 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:36.495248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:36.495345 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:36.495621 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:36.495623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:36.495631 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:36.495772 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.495778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.496838 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.496841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.497489 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:36.497497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.497930 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.497933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:36.498019 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.498055 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.498357 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.498365 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:36.498425 (+ 60us) client_negotiation.cc:240] Negotiation successful
0621 07:56:36.498497 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":98,"threads_started":1}
I20260621 07:56:36.498627 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:36.494939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:43139 (local address 127.11.223.130:46667)
0621 07:56:36.495055 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:36.495059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:36.495328 (+ 269us) server_negotiation.cc:407] Connection header received
0621 07:56:36.495429 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:36.495432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:36.495501 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:36.495580 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:36.495912 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.496592 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.497602 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:36.497814 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:36.497920 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:36.498173 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:36.498267 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:36.498342 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:36.498434 (+ 92us) server_negotiation.cc:299] Negotiation successful
0621 07:56:36.498480 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 07:56:36.524358 13890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:36.524544 13890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:36.527037 13890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Sending DeleteTablet for 3 replicas of tablet fe4772e3e0244a0ebdc0347b06dd1ca3
I20260621 07:56:36.527938 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet fe4772e3e0244a0ebdc0347b06dd1ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:36 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260621 07:56:36.528388 14444 tablet_replica.cc:330] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: stopping tablet replica
I20260621 07:56:36.528393 14309 tablet_service.cc:1514] Processing DeleteTablet for tablet fe4772e3e0244a0ebdc0347b06dd1ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:36 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50638
I20260621 07:56:36.528756 14444 raft_consensus.cc:2229] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:56:36.528869 14029 tablet_service.cc:1514] Processing DeleteTablet for tablet fe4772e3e0244a0ebdc0347b06dd1ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:36 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35346
I20260621 07:56:36.529141 14444 raft_consensus.cc:2258] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:36.529932 13890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:36.530366 13890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:36.530673 14444 ts_tablet_manager.cc:1905] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:36.533756 14444 ts_tablet_manager.cc:1918] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:36.534264 14445 tablet_replica.cc:330] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: stopping tablet replica
I20260621 07:56:36.534565 14444 log.cc:1196] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/fe4772e3e0244a0ebdc0347b06dd1ca3
I20260621 07:56:36.534806 14445 raft_consensus.cc:2229] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:36.535216 14444 ts_tablet_manager.cc:1939] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 35be65c2d2b5417b847bfb3d635afc63: Deleting consensus metadata
I20260621 07:56:36.535465 14445 raft_consensus.cc:2258] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:36.536223 14029 tablet_service.cc:1467] Processing CreateTablet for tablet 1a0d360249c04d5fba969dcd736d9981 (DEFAULT_TABLE table=test-table [id=a1e01a3fa09f4ae4beb9c8a81922e6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:36.536473 14446 tablet_replica.cc:330] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: stopping tablet replica
I20260621 07:56:36.536502 14029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a0d360249c04d5fba969dcd736d9981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.536715 14446 raft_consensus.cc:2229] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:36.536862 14446 raft_consensus.cc:2258] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:36.538765 14400 tablet_bootstrap.cc:492] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Bootstrap starting.
I20260621 07:56:36.538905 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 1a0d360249c04d5fba969dcd736d9981 (DEFAULT_TABLE table=test-table [id=a1e01a3fa09f4ae4beb9c8a81922e6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:36.539278 14309 tablet_service.cc:1467] Processing CreateTablet for tablet 1a0d360249c04d5fba969dcd736d9981 (DEFAULT_TABLE table=test-table [id=a1e01a3fa09f4ae4beb9c8a81922e6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:36.539645 14400 tablet_bootstrap.cc:654] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.539759 14309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a0d360249c04d5fba969dcd736d9981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.541838 14400 tablet_bootstrap.cc:492] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: No bootstrap required, opened a new log
I20260621 07:56:36.541904 14400 ts_tablet_manager.cc:1397] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260621 07:56:36.542138 14399 tablet_bootstrap.cc:492] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Bootstrap starting.
I20260621 07:56:36.542253 14400 raft_consensus.cc:348] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.542347 14400 raft_consensus.cc:374] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.542375 14400 raft_consensus.cc:729] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.542479 14400 consensus_queue.cc:260] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.542713 14400 ts_tablet_manager.cc:1428] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:36.542968 14399 tablet_bootstrap.cc:654] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.544658 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a0d360249c04d5fba969dcd736d9981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:36.545356 14446 ts_tablet_manager.cc:1905] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:36.546826 14398 tablet_bootstrap.cc:492] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Bootstrap starting.
I20260621 07:56:36.547647 14398 tablet_bootstrap.cc:654] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:36.547843 14446 ts_tablet_manager.cc:1918] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:36.547919 14446 log.cc:1196] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/fe4772e3e0244a0ebdc0347b06dd1ca3
I20260621 07:56:36.548254 14446 ts_tablet_manager.cc:1939] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 9d60398cd64341749f57d78e115d2590: Deleting consensus metadata
I20260621 07:56:36.548646 13878 catalog_manager.cc:4841] TS 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667): tablet fe4772e3e0244a0ebdc0347b06dd1ca3 (table test-table [id=720e2e600459408487ec2a2b65119701]) successfully deleted
I20260621 07:56:36.548966 14445 ts_tablet_manager.cc:1905] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:36.549373 13878 catalog_manager.cc:4841] TS 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071): tablet fe4772e3e0244a0ebdc0347b06dd1ca3 (table test-table [id=720e2e600459408487ec2a2b65119701]) successfully deleted
I20260621 07:56:36.550160 14398 tablet_bootstrap.cc:492] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: No bootstrap required, opened a new log
I20260621 07:56:36.550237 14398 ts_tablet_manager.cc:1397] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260621 07:56:36.550616 14398 raft_consensus.cc:348] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.550724 14398 raft_consensus.cc:374] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.550755 14398 raft_consensus.cc:729] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.550861 14398 consensus_queue.cc:260] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.551098 14398 ts_tablet_manager.cc:1428] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:36.552332 14399 tablet_bootstrap.cc:492] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: No bootstrap required, opened a new log
I20260621 07:56:36.552510 14399 ts_tablet_manager.cc:1397] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260621 07:56:36.553279 14399 raft_consensus.cc:348] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.553454 14399 raft_consensus.cc:374] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:36.553512 14399 raft_consensus.cc:729] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f055617ecd407ea9902180927f81e3, State: Initialized, Role: FOLLOWER
I20260621 07:56:36.553635 14399 consensus_queue.cc:260] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.553874 14399 ts_tablet_manager.cc:1428] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:36.556944 14445 ts_tablet_manager.cc:1918] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:36.557111 14445 log.cc:1196] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/fe4772e3e0244a0ebdc0347b06dd1ca3
I20260621 07:56:36.557924 14445 ts_tablet_manager.cc:1939] T fe4772e3e0244a0ebdc0347b06dd1ca3 P 30f055617ecd407ea9902180927f81e3: Deleting consensus metadata
I20260621 07:56:36.559075 13877 catalog_manager.cc:4841] TS 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085): tablet fe4772e3e0244a0ebdc0347b06dd1ca3 (table test-table [id=720e2e600459408487ec2a2b65119701]) successfully deleted
W20260621 07:56:36.560474 14236 tablet.cc:2369] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:36.708976 14405 raft_consensus.cc:482] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:36.709136 14405 raft_consensus.cc:504] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.709575 14405 leader_election.cc:288] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.710035 14328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a0d360249c04d5fba969dcd736d9981" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3" is_pre_election: true
I20260621 07:56:36.710075 14049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a0d360249c04d5fba969dcd736d9981" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d60398cd64341749f57d78e115d2590" is_pre_election: true
I20260621 07:56:36.710248 14328 raft_consensus.cc:2454] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 0.
I20260621 07:56:36.710258 14049 raft_consensus.cc:2454] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 0.
I20260621 07:56:36.710616 14122 leader_election.cc:302] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35be65c2d2b5417b847bfb3d635afc63, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:36.710816 14405 raft_consensus.cc:2781] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:36.710878 14405 raft_consensus.cc:482] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:36.710907 14405 raft_consensus.cc:3037] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.711910 14405 raft_consensus.cc:504] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.712302 14405 leader_election.cc:288] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 election: Requested vote from peers 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:36.712661 14049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a0d360249c04d5fba969dcd736d9981" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d60398cd64341749f57d78e115d2590"
I20260621 07:56:36.712718 14328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a0d360249c04d5fba969dcd736d9981" candidate_uuid: "35be65c2d2b5417b847bfb3d635afc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3"
I20260621 07:56:36.712803 14049 raft_consensus.cc:3037] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.712853 14328 raft_consensus.cc:3037] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:36.713765 14328 raft_consensus.cc:2454] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 1.
I20260621 07:56:36.713774 14049 raft_consensus.cc:2454] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35be65c2d2b5417b847bfb3d635afc63 in term 1.
I20260621 07:56:36.714068 14121 leader_election.cc:302] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f055617ecd407ea9902180927f81e3, 35be65c2d2b5417b847bfb3d635afc63; no voters:
I20260621 07:56:36.714254 14405 raft_consensus.cc:2781] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:36.714347 14405 raft_consensus.cc:686] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 LEADER]: Becoming Leader. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Running, Role: LEADER
I20260621 07:56:36.714494 14405 consensus_queue.cc:237] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:36.716214 13890 catalog_manager.cc:5495] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "35be65c2d2b5417b847bfb3d635afc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } health_report { overall_health: UNKNOWN } } }
I20260621 07:56:36.738672 14329 raft_consensus.cc:1261] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Refusing update from remote peer 35be65c2d2b5417b847bfb3d635afc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:36.738785 14049 raft_consensus.cc:1261] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Refusing update from remote peer 35be65c2d2b5417b847bfb3d635afc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:36.739137 14404 consensus_queue.cc:1035] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:36.739266 14405 consensus_queue.cc:1035] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:36.743980 14427 mvcc.cc:205] Tried to move back new op lower bound from 7299189132238626816 to 7299189132144185344. Current Snapshot: MvccSnapshot[applied={T|T < 7299189132238626816}]
W20260621 07:56:36.751446 14428 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260621 07:56:38.298617 13891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:38.298758 13891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:38.300879 13891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Sending DeleteTablet for 3 replicas of tablet 1a0d360249c04d5fba969dcd736d9981
I20260621 07:56:38.301437 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a0d360249c04d5fba969dcd736d9981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:38 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260621 07:56:38.301476 14029 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a0d360249c04d5fba969dcd736d9981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:38 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35346
I20260621 07:56:38.301509 14309 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a0d360249c04d5fba969dcd736d9981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:38 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50638
I20260621 07:56:38.301818 14477 tablet_replica.cc:330] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: stopping tablet replica
I20260621 07:56:38.301839 14478 tablet_replica.cc:330] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: stopping tablet replica
I20260621 07:56:38.301939 14477 raft_consensus.cc:2229] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:38.301939 14478 raft_consensus.cc:2229] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:38.302040 14478 raft_consensus.cc:2258] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:38.302067 14477 raft_consensus.cc:2258] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:38.302647 14477 ts_tablet_manager.cc:1905] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:38.302659 14478 ts_tablet_manager.cc:1905] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:38.304955 14476 tablet_replica.cc:330] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: stopping tablet replica
I20260621 07:56:38.305042 14477 ts_tablet_manager.cc:1918] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 07:56:38.305042 14478 ts_tablet_manager.cc:1918] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 07:56:38.305166 14477 log.cc:1196] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/1a0d360249c04d5fba969dcd736d9981
I20260621 07:56:38.305222 14476 raft_consensus.cc:2229] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:56:38.305419 14477 ts_tablet_manager.cc:1939] T 1a0d360249c04d5fba969dcd736d9981 P 9d60398cd64341749f57d78e115d2590: Deleting consensus metadata
I20260621 07:56:38.305434 14476 raft_consensus.cc:2258] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:38.305711 14478 log.cc:1196] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/1a0d360249c04d5fba969dcd736d9981
I20260621 07:56:38.305984 13878 catalog_manager.cc:4841] TS 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071): tablet 1a0d360249c04d5fba969dcd736d9981 (table test-table [id=a1e01a3fa09f4ae4beb9c8a81922e6f6]) successfully deleted
I20260621 07:56:38.306033 14478 ts_tablet_manager.cc:1939] T 1a0d360249c04d5fba969dcd736d9981 P 30f055617ecd407ea9902180927f81e3: Deleting consensus metadata
I20260621 07:56:38.306248 14476 ts_tablet_manager.cc:1905] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:38.306744 13877 catalog_manager.cc:4841] TS 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085): tablet 1a0d360249c04d5fba969dcd736d9981 (table test-table [id=a1e01a3fa09f4ae4beb9c8a81922e6f6]) successfully deleted
I20260621 07:56:38.308760 14476 ts_tablet_manager.cc:1918] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 07:56:38.308822 14476 log.cc:1196] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/1a0d360249c04d5fba969dcd736d9981
I20260621 07:56:38.309077 14476 ts_tablet_manager.cc:1939] T 1a0d360249c04d5fba969dcd736d9981 P 35be65c2d2b5417b847bfb3d635afc63: Deleting consensus metadata
I20260621 07:56:38.309659 13878 catalog_manager.cc:4841] TS 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667): tablet 1a0d360249c04d5fba969dcd736d9981 (table test-table [id=a1e01a3fa09f4ae4beb9c8a81922e6f6]) successfully deleted
Jun 21 07:56:38 dist-test-slave-0pn3 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 07:56:38 dist-test-slave-0pn3 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028598, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:38.331287 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.318910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36606 (local address 127.11.223.190:39461)
0621 07:56:38.319161 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.319167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.319179 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:56:38.319294 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.319296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.319335 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.319407 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:38.320006 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.320639 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.321369 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.321566 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.321635 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.328198 (+ 6563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:38.328201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:38.328203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:38.328224 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:38.329787 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:38.330173 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:38.330175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:38.330177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:38.330246 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:38.330461 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:38.330463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:38.330464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:38.330637 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:38.330742 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.330952 (+ 210us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.331097 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":126,"threads_started":1}
I20260621 07:56:38.333648 13891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:38.334084 13891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:38.338404 14029 tablet_service.cc:1467] Processing CreateTablet for tablet 6314ab3503a54fdfbc584c46c6f72484 (DEFAULT_TABLE table=test-table [id=43ad6a7f33a3497dad6b963bcd58b5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:38.338434 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 6314ab3503a54fdfbc584c46c6f72484 (DEFAULT_TABLE table=test-table [id=43ad6a7f33a3497dad6b963bcd58b5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:38.338717 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6314ab3503a54fdfbc584c46c6f72484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:38.338724 14029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6314ab3503a54fdfbc584c46c6f72484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:38.338685 14309 tablet_service.cc:1467] Processing CreateTablet for tablet 6314ab3503a54fdfbc584c46c6f72484 (DEFAULT_TABLE table=test-table [id=43ad6a7f33a3497dad6b963bcd58b5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:38.338922 14309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6314ab3503a54fdfbc584c46c6f72484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:38.340776 14487 tablet_bootstrap.cc:492] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Bootstrap starting.
I20260621 07:56:38.341177 14488 tablet_bootstrap.cc:492] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Bootstrap starting.
I20260621 07:56:38.341777 14487 tablet_bootstrap.cc:654] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:38.342109 14488 tablet_bootstrap.cc:654] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:38.342685 14489 tablet_bootstrap.cc:492] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Bootstrap starting.
I20260621 07:56:38.342893 14487 tablet_bootstrap.cc:492] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: No bootstrap required, opened a new log
I20260621 07:56:38.342976 14487 ts_tablet_manager.cc:1397] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:38.343364 14487 raft_consensus.cc:348] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.343488 14487 raft_consensus.cc:374] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:38.343539 14487 raft_consensus.cc:729] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Initialized, Role: FOLLOWER
I20260621 07:56:38.343550 14489 tablet_bootstrap.cc:654] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:38.343689 14487 consensus_queue.cc:260] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.344071 14487 ts_tablet_manager.cc:1428] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:38.344493 14488 tablet_bootstrap.cc:492] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: No bootstrap required, opened a new log
I20260621 07:56:38.344609 14488 ts_tablet_manager.cc:1397] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260621 07:56:38.344971 14488 raft_consensus.cc:348] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.345081 14488 raft_consensus.cc:374] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:38.345152 14488 raft_consensus.cc:729] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Initialized, Role: FOLLOWER
I20260621 07:56:38.345299 14488 consensus_queue.cc:260] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.345583 14488 ts_tablet_manager.cc:1428] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260621 07:56:38.346053 14489 tablet_bootstrap.cc:492] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: No bootstrap required, opened a new log
I20260621 07:56:38.346138 14489 ts_tablet_manager.cc:1397] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260621 07:56:38.346871 14489 raft_consensus.cc:348] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.346990 14489 raft_consensus.cc:374] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:38.347069 14489 raft_consensus.cc:729] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f055617ecd407ea9902180927f81e3, State: Initialized, Role: FOLLOWER
I20260621 07:56:38.347203 14489 consensus_queue.cc:260] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.347570 14489 ts_tablet_manager.cc:1428] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:38.441848 14493 raft_consensus.cc:482] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:38.441994 14493 raft_consensus.cc:504] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.443117 14493 leader_election.cc:288] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071), 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667)
I20260621 07:56:38.448171 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.443867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:32941 (local address 127.11.223.130:46667)
0621 07:56:38.444178 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.444186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.444198 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:56:38.444246 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.444248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.444309 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.444413 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:38.444612 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.445324 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.446892 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.447479 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.447597 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.447659 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.447971 (+ 312us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.448039 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":114,"threads_started":1}
I20260621 07:56:38.448175 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.443382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:46625 (local address 127.11.223.129:46071)
0621 07:56:38.443656 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.443663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.443731 (+ 68us) server_negotiation.cc:407] Connection header received
0621 07:56:38.443879 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.443883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.443948 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.444046 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:38.444336 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.445160 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.446892 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.447485 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.447597 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.447645 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.447971 (+ 326us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.448039 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":123,"threads_started":1}
I20260621 07:56:38.448225 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.443338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.11.223.131:46625)
0621 07:56:38.443623 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:38.443641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:38.443758 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:38.444067 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:38.444069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:38.444082 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:38.444214 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.444220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.445281 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.445283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.446772 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.446782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.447608 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.447611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.447810 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.447865 (+ 55us) client_negotiation.cc:769] Sending connection context
0621 07:56:38.447941 (+ 76us) client_negotiation.cc:240] Negotiation successful
0621 07:56:38.448053 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":296,"thread_start_us":129,"threads_started":1}
I20260621 07:56:38.448228 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.443740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:46667 (local address 127.11.223.131:32941)
0621 07:56:38.443974 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:38.443984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:38.444076 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:38.444418 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:38.444420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:38.444434 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:38.444508 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.444513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.445423 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.445425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.446772 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.446782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.447608 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.447611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.447785 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.447866 (+ 81us) client_negotiation.cc:769] Sending connection context
0621 07:56:38.447941 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:56:38.448053 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":149,"thread_start_us":80,"threads_started":1}
I20260621 07:56:38.448662 14049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6314ab3503a54fdfbc584c46c6f72484" candidate_uuid: "30f055617ecd407ea9902180927f81e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d60398cd64341749f57d78e115d2590" is_pre_election: true
I20260621 07:56:38.448817 14049 raft_consensus.cc:2454] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f055617ecd407ea9902180927f81e3 in term 0.
I20260621 07:56:38.449151 14262 leader_election.cc:302] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f055617ecd407ea9902180927f81e3, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:38.449190 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6314ab3503a54fdfbc584c46c6f72484" candidate_uuid: "30f055617ecd407ea9902180927f81e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35be65c2d2b5417b847bfb3d635afc63" is_pre_election: true
I20260621 07:56:38.449321 14189 raft_consensus.cc:2454] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30f055617ecd407ea9902180927f81e3 in term 0.
I20260621 07:56:38.449421 14493 raft_consensus.cc:2781] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:38.449505 14493 raft_consensus.cc:482] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:38.449573 14493 raft_consensus.cc:3037] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:38.450517 14493 raft_consensus.cc:504] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.450819 14493 leader_election.cc:288] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [CANDIDATE]: Term 1 election: Requested vote from peers 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071), 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667)
I20260621 07:56:38.451134 14049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6314ab3503a54fdfbc584c46c6f72484" candidate_uuid: "30f055617ecd407ea9902180927f81e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d60398cd64341749f57d78e115d2590"
I20260621 07:56:38.451247 14049 raft_consensus.cc:3037] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:38.451232 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6314ab3503a54fdfbc584c46c6f72484" candidate_uuid: "30f055617ecd407ea9902180927f81e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35be65c2d2b5417b847bfb3d635afc63"
I20260621 07:56:38.451349 14189 raft_consensus.cc:3037] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:38.452419 14049 raft_consensus.cc:2454] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f055617ecd407ea9902180927f81e3 in term 1.
I20260621 07:56:38.452466 14189 raft_consensus.cc:2454] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30f055617ecd407ea9902180927f81e3 in term 1.
I20260621 07:56:38.452670 14262 leader_election.cc:302] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f055617ecd407ea9902180927f81e3, 35be65c2d2b5417b847bfb3d635afc63; no voters:
I20260621 07:56:38.452826 14493 raft_consensus.cc:2781] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:38.453042 14493 raft_consensus.cc:686] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 1 LEADER]: Becoming Leader. State: Replica: 30f055617ecd407ea9902180927f81e3, State: Running, Role: LEADER
I20260621 07:56:38.453270 14493 consensus_queue.cc:237] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.455217 13890 catalog_manager.cc:5495] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30f055617ecd407ea9902180927f81e3 (127.11.223.131). New cstate: current_term: 1 leader_uuid: "30f055617ecd407ea9902180927f81e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } health_report { overall_health: HEALTHY } } }
I20260621 07:56:38.470161 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.466782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.11.223.131:38085)
0621 07:56:38.467018 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.467024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.467035 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:38.467089 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.467091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.467131 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.467245 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:38.467437 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.468020 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.468865 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.469081 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.469163 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.469363 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.469769 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.469851 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.469902 (+ 51us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.469940 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":122,"threads_started":1}
I20260621 07:56:38.479041 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.476036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:41115 (local address 127.11.223.129:46071)
0621 07:56:38.476151 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.476156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.476358 (+ 202us) server_negotiation.cc:407] Connection header received
0621 07:56:38.476505 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.476509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.476552 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.476625 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:38.476854 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.477312 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.478063 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.478254 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.478353 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.478639 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.478705 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.478783 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.478876 (+ 93us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.478920 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 07:56:38.479054 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.475939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.11.223.131:41115)
0621 07:56:38.476257 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:38.476268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:38.476406 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:38.476637 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:38.476640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:38.476647 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:38.476757 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.476762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.477414 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.477417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.477955 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.477962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.478354 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.478375 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.478471 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.478506 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.478792 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.478800 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:38.478857 (+ 57us) client_negotiation.cc:240] Negotiation successful
0621 07:56:38.478928 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":88,"threads_started":1}
I20260621 07:56:38.483896 14049 raft_consensus.cc:1261] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Refusing update from remote peer 30f055617ecd407ea9902180927f81e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:38.483999 14189 raft_consensus.cc:1261] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Refusing update from remote peer 30f055617ecd407ea9902180927f81e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:38.484361 14498 consensus_queue.cc:1035] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:38.484556 14493 consensus_queue.cc:1035] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:38.500474 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.497241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38085 (local address 127.11.223.129:37581)
0621 07:56:38.497532 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:38.497545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:38.497658 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:38.497891 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:38.497894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:38.497901 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:38.497987 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.497992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.498662 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.498668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.499249 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.499257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.499704 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.499707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.499789 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.499816 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.500121 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.500136 (+ 15us) client_negotiation.cc:769] Sending connection context
0621 07:56:38.500223 (+ 87us) client_negotiation.cc:240] Negotiation successful
0621 07:56:38.500298 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":91,"threads_started":1}
I20260621 07:56:38.500490 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.497318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:37581 (local address 127.11.223.131:38085)
0621 07:56:38.497434 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.497438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.497623 (+ 185us) server_negotiation.cc:407] Connection header received
0621 07:56:38.497754 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.497757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.497795 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.497882 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:38.498092 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.498556 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.499364 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.499596 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.499706 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.499931 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.499997 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.500108 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.500309 (+ 201us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.500382 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 07:56:38.517009 13891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:38.517171 13891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:38.519089 13891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Sending DeleteTablet for 3 replicas of tablet 6314ab3503a54fdfbc584c46c6f72484
I20260621 07:56:38.519552 14029 tablet_service.cc:1514] Processing DeleteTablet for tablet 6314ab3503a54fdfbc584c46c6f72484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:38 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35346
I20260621 07:56:38.519668 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet 6314ab3503a54fdfbc584c46c6f72484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:38 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260621 07:56:38.519829 14476 tablet_replica.cc:330] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: stopping tablet replica
I20260621 07:56:38.519833 14477 tablet_replica.cc:330] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: stopping tablet replica
I20260621 07:56:38.519824 14309 tablet_service.cc:1514] Processing DeleteTablet for tablet 6314ab3503a54fdfbc584c46c6f72484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:38 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50638
I20260621 07:56:38.519940 14476 raft_consensus.cc:2229] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:38.519960 14478 tablet_replica.cc:330] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: stopping tablet replica
I20260621 07:56:38.519949 14477 raft_consensus.cc:2229] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:38.520041 14477 raft_consensus.cc:2258] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:38.520040 14478 raft_consensus.cc:2229] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:56:38.520045 14476 raft_consensus.cc:2258] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:38.520284 14478 raft_consensus.cc:2258] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:38.520730 14477 ts_tablet_manager.cc:1905] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:38.520802 14476 ts_tablet_manager.cc:1905] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:38.523109 14477 ts_tablet_manager.cc:1918] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:38.523165 14477 log.cc:1196] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/6314ab3503a54fdfbc584c46c6f72484
I20260621 07:56:38.523368 14476 ts_tablet_manager.cc:1918] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:38.523422 14477 ts_tablet_manager.cc:1939] T 6314ab3503a54fdfbc584c46c6f72484 P 9d60398cd64341749f57d78e115d2590: Deleting consensus metadata
I20260621 07:56:38.523445 14476 log.cc:1196] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/6314ab3503a54fdfbc584c46c6f72484
I20260621 07:56:38.523712 14476 ts_tablet_manager.cc:1939] T 6314ab3503a54fdfbc584c46c6f72484 P 35be65c2d2b5417b847bfb3d635afc63: Deleting consensus metadata
I20260621 07:56:38.523953 13878 catalog_manager.cc:4841] TS 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071): tablet 6314ab3503a54fdfbc584c46c6f72484 (table test-table [id=43ad6a7f33a3497dad6b963bcd58b5c5]) successfully deleted
I20260621 07:56:38.524304 13878 catalog_manager.cc:4841] TS 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667): tablet 6314ab3503a54fdfbc584c46c6f72484 (table test-table [id=43ad6a7f33a3497dad6b963bcd58b5c5]) successfully deleted
I20260621 07:56:38.524617 13891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260621 07:56:38.524734 14478 ts_tablet_manager.cc:1905] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260621 07:56:38.524979 13891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:38.526795 14478 ts_tablet_manager.cc:1918] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:56:38.526865 14478 log.cc:1196] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/6314ab3503a54fdfbc584c46c6f72484
I20260621 07:56:38.527096 14478 ts_tablet_manager.cc:1939] T 6314ab3503a54fdfbc584c46c6f72484 P 30f055617ecd407ea9902180927f81e3: Deleting consensus metadata
I20260621 07:56:38.527607 13877 catalog_manager.cc:4841] TS 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085): tablet 6314ab3503a54fdfbc584c46c6f72484 (table test-table [id=43ad6a7f33a3497dad6b963bcd58b5c5]) successfully deleted
I20260621 07:56:38.529088 14029 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa8de463b1746838489c3a87cd8f98d (DEFAULT_TABLE table=test-table [id=cb2c71aa389149c38fbde11185bfc2b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:38.529135 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa8de463b1746838489c3a87cd8f98d (DEFAULT_TABLE table=test-table [id=cb2c71aa389149c38fbde11185bfc2b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:38.529263 14309 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa8de463b1746838489c3a87cd8f98d (DEFAULT_TABLE table=test-table [id=cb2c71aa389149c38fbde11185bfc2b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:38.529337 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa8de463b1746838489c3a87cd8f98d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:38.529337 14029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa8de463b1746838489c3a87cd8f98d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:38.529470 14309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa8de463b1746838489c3a87cd8f98d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:38.531409 14488 tablet_bootstrap.cc:492] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590: Bootstrap starting.
I20260621 07:56:38.531452 14487 tablet_bootstrap.cc:492] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Bootstrap starting.
I20260621 07:56:38.532099 14488 tablet_bootstrap.cc:654] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:38.532155 14487 tablet_bootstrap.cc:654] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:38.532716 14489 tablet_bootstrap.cc:492] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Bootstrap starting.
I20260621 07:56:38.533248 14488 tablet_bootstrap.cc:492] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590: No bootstrap required, opened a new log
I20260621 07:56:38.533349 14488 ts_tablet_manager.cc:1397] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:38.533525 14489 tablet_bootstrap.cc:654] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:38.533749 14488 raft_consensus.cc:348] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.533862 14488 raft_consensus.cc:374] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:38.533900 14488 raft_consensus.cc:729] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Initialized, Role: FOLLOWER
I20260621 07:56:38.534001 14488 consensus_queue.cc:260] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.534214 14488 ts_tablet_manager.cc:1428] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:38.534973 14487 tablet_bootstrap.cc:492] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: No bootstrap required, opened a new log
I20260621 07:56:38.535043 14487 ts_tablet_manager.cc:1397] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260621 07:56:38.535400 14487 raft_consensus.cc:348] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.535492 14487 raft_consensus.cc:374] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:38.535522 14487 raft_consensus.cc:729] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35be65c2d2b5417b847bfb3d635afc63, State: Initialized, Role: FOLLOWER
I20260621 07:56:38.535635 14487 consensus_queue.cc:260] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.535849 14487 ts_tablet_manager.cc:1428] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:56:38.536511 14489 tablet_bootstrap.cc:492] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: No bootstrap required, opened a new log
I20260621 07:56:38.536597 14489 ts_tablet_manager.cc:1397] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260621 07:56:38.536962 14489 raft_consensus.cc:348] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.537050 14489 raft_consensus.cc:374] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:38.537079 14489 raft_consensus.cc:729] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f055617ecd407ea9902180927f81e3, State: Initialized, Role: FOLLOWER
I20260621 07:56:38.537177 14489 consensus_queue.cc:260] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.537412 14489 ts_tablet_manager.cc:1428] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:38.679270 14462 raft_consensus.cc:482] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:38.679397 14462 raft_consensus.cc:504] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.679883 14462 leader_election.cc:288] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:38.680364 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa8de463b1746838489c3a87cd8f98d" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35be65c2d2b5417b847bfb3d635afc63" is_pre_election: true
I20260621 07:56:38.680398 14329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa8de463b1746838489c3a87cd8f98d" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3" is_pre_election: true
I20260621 07:56:38.680505 14189 raft_consensus.cc:2454] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 0.
I20260621 07:56:38.680572 14329 raft_consensus.cc:2454] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 0.
I20260621 07:56:38.680827 13982 leader_election.cc:302] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35be65c2d2b5417b847bfb3d635afc63, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:38.681036 14462 raft_consensus.cc:2781] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:38.681103 14462 raft_consensus.cc:482] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:38.681130 14462 raft_consensus.cc:3037] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:38.682379 14462 raft_consensus.cc:504] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.682824 14462 leader_election.cc:288] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 election: Requested vote from peers 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667), 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085)
I20260621 07:56:38.683005 14329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa8de463b1746838489c3a87cd8f98d" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f055617ecd407ea9902180927f81e3"
I20260621 07:56:38.683054 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa8de463b1746838489c3a87cd8f98d" candidate_uuid: "9d60398cd64341749f57d78e115d2590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35be65c2d2b5417b847bfb3d635afc63"
I20260621 07:56:38.683112 14329 raft_consensus.cc:3037] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:38.683197 14189 raft_consensus.cc:3037] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:38.684000 14329 raft_consensus.cc:2454] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 1.
I20260621 07:56:38.684157 14189 raft_consensus.cc:2454] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d60398cd64341749f57d78e115d2590 in term 1.
I20260621 07:56:38.684249 13981 leader_election.cc:302] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f055617ecd407ea9902180927f81e3, 9d60398cd64341749f57d78e115d2590; no voters:
I20260621 07:56:38.684424 14462 raft_consensus.cc:2781] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:38.684616 14462 raft_consensus.cc:686] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [term 1 LEADER]: Becoming Leader. State: Replica: 9d60398cd64341749f57d78e115d2590, State: Running, Role: LEADER
I20260621 07:56:38.684773 14462 consensus_queue.cc:237] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } }
I20260621 07:56:38.686236 13890 catalog_manager.cc:5495] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d60398cd64341749f57d78e115d2590 (127.11.223.129). New cstate: current_term: 1 leader_uuid: "9d60398cd64341749f57d78e115d2590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d60398cd64341749f57d78e115d2590" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 46071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 } health_report { overall_health: UNKNOWN } } }
I20260621 07:56:38.729758 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.727106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.11.223.129:46071)
0621 07:56:38.727221 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.727225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.727236 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:38.727342 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.727344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.727380 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.727429 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:38.727617 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.728097 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.728935 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.729132 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.729211 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.729399 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.729455 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.729506 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.729594 (+ 88us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.729632 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:56:38.737493 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.734645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.11.223.129:33629)
0621 07:56:38.734763 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:38.734776 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:38.734869 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:38.735072 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:38.735075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:38.735081 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:56:38.735164 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.735169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.735847 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.735851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.736468 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:38.736477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.736867 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.736870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:38.736958 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.736981 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.737239 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.737247 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:38.737304 (+ 57us) client_negotiation.cc:240] Negotiation successful
0621 07:56:38.737377 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":8}
I20260621 07:56:38.737493 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:38.734784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:33629 (local address 127.11.223.129:46071)
0621 07:56:38.734882 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:38.734886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:38.734900 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:56:38.734937 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:38.734942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:38.734975 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:38.735057 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:38.735323 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.735751 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.736568 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:38.736757 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:38.736867 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:38.737099 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:38.737157 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:38.737227 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:38.737311 (+ 84us) server_negotiation.cc:299] Negotiation successful
0621 07:56:38.737353 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":46}
I20260621 07:56:38.742281 14329 raft_consensus.cc:1261] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Refusing update from remote peer 9d60398cd64341749f57d78e115d2590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:38.742283 14189 raft_consensus.cc:1261] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Refusing update from remote peer 9d60398cd64341749f57d78e115d2590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:38.742611 14462 consensus_queue.cc:1035] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f055617ecd407ea9902180927f81e3" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:38.742772 14439 consensus_queue.cc:1035] T 2fa8de463b1746838489c3a87cd8f98d P 9d60398cd64341749f57d78e115d2590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35be65c2d2b5417b847bfb3d635afc63" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 07:56:38.754585 14455 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260621 07:56:40.304473 13890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:40.304612 13890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:40.306643 13890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 948a8909d01e4989b12ebafc8e9974d7: Sending DeleteTablet for 3 replicas of tablet 2fa8de463b1746838489c3a87cd8f98d
I20260621 07:56:40.307094 14028 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa8de463b1746838489c3a87cd8f98d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:40 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35346
I20260621 07:56:40.307180 14309 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa8de463b1746838489c3a87cd8f98d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:40 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50638
I20260621 07:56:40.307180 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa8de463b1746838489c3a87cd8f98d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:40 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260621 07:56:40.307466 14539 tablet_replica.cc:330] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: stopping tablet replica
I20260621 07:56:40.307611 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13964
I20260621 07:56:40.308346 14539 raft_consensus.cc:2229] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:40.308482 14539 raft_consensus.cc:2258] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:40.308709 14540 tablet_replica.cc:330] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: stopping tablet replica
I20260621 07:56:40.308851 14540 raft_consensus.cc:2229] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:40.308951 14540 raft_consensus.cc:2258] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:40.309568 14540 ts_tablet_manager.cc:1905] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:40.309568 14539 ts_tablet_manager.cc:1905] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:40.312214 14540 ts_tablet_manager.cc:1918] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 07:56:40.312214 14539 ts_tablet_manager.cc:1918] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260621 07:56:40.312299 14539 log.cc:1196] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/2fa8de463b1746838489c3a87cd8f98d
I20260621 07:56:40.312299 14540 log.cc:1196] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/2fa8de463b1746838489c3a87cd8f98d
I20260621 07:56:40.312621 14540 ts_tablet_manager.cc:1939] T 2fa8de463b1746838489c3a87cd8f98d P 30f055617ecd407ea9902180927f81e3: Deleting consensus metadata
I20260621 07:56:40.312630 14539 ts_tablet_manager.cc:1939] T 2fa8de463b1746838489c3a87cd8f98d P 35be65c2d2b5417b847bfb3d635afc63: Deleting consensus metadata
I20260621 07:56:40.313221 13877 catalog_manager.cc:4841] TS 30f055617ecd407ea9902180927f81e3 (127.11.223.131:38085): tablet 2fa8de463b1746838489c3a87cd8f98d (table test-table [id=cb2c71aa389149c38fbde11185bfc2b7]) successfully deleted
I20260621 07:56:40.313221 13878 catalog_manager.cc:4841] TS 35be65c2d2b5417b847bfb3d635afc63 (127.11.223.130:46667): tablet 2fa8de463b1746838489c3a87cd8f98d (table test-table [id=cb2c71aa389149c38fbde11185bfc2b7]) successfully deleted
W20260621 07:56:40.315528 13878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:46071 (error 108)
I20260621 07:56:40.315717 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 14104
I20260621 07:56:40.317724 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:40.317383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:46071 (local address 127.0.0.1:35376)
0621 07:56:40.317590 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:40.317652 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:46071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
W20260621 07:56:40.317847 13878 catalog_manager.cc:4584] TS 9d60398cd64341749f57d78e115d2590 (127.11.223.129:46071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fa8de463b1746838489c3a87cd8f98d: Network error: Client connection negotiation failed: client connection to 127.11.223.129:46071: connect: Connection refused (error 111)
I20260621 07:56:40.322180 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 14244
I20260621 07:56:40.328789 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 13860
2026-06-21T07:56:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6008 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:40 dist-test-slave-0pn3 krb5kdc[14545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:40 dist-test-slave-0pn3 krb5kdc[14545](info): set up 2 sockets
Jun 21 07:56:40 dist-test-slave-0pn3 krb5kdc[14545](info): commencing operation
krb5kdc: starting...
W20260621 07:56:41.332768 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:41 dist-test-slave-0pn3 krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:41.441066 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43969
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:36593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:41.540076 14558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:41.540305 14558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:41.540402 14558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:41.544051 14558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:41.544121 14558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:41.544174 14558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:41.544229 14558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:41.544263 14558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:41.548871 14558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:36593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:41.549729 14558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:41.550606 14558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:56:41.558382 14558 server_base.cc:1034] running on GCE node
W20260621 07:56:41.558339 14564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:41.558350 14563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:41.558351 14566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:41.559126 14558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:41.560092 14558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:41.561271 14558 hybrid_clock.cc:648] HybridClock initialized: now 1782028601561263 us; error 41 us; skew 500 ppm
Jun 21 07:56:41 dist-test-slave-0pn3 krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:41.563727 14558 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:41.564554 14558 webserver.cc:460] Webserver started at http://127.11.223.190:34991/ using document root <none> and password file <none>
I20260621 07:56:41.565052 14558 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:41.565135 14558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:41.565335 14558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:41.567123 14558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "06fd55eb0e3c422d81dc0dfe0da942b0"
format_stamp: "Formatted at 2026-06-21 07:56:41 on dist-test-slave-0pn3"
server_key: "4d547669f24b3d0007ce62d9b2e15400"
server_key_iv: "0174935fce21d26db014a47cab89ed6e"
server_key_version: "encryptionkey@0"
I20260621 07:56:41.567664 14558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "06fd55eb0e3c422d81dc0dfe0da942b0"
format_stamp: "Formatted at 2026-06-21 07:56:41 on dist-test-slave-0pn3"
server_key: "4d547669f24b3d0007ce62d9b2e15400"
server_key_iv: "0174935fce21d26db014a47cab89ed6e"
server_key_version: "encryptionkey@0"
I20260621 07:56:41.571724 14558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260621 07:56:41.574429 14573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:41.575555 14558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:41.575709 14558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "06fd55eb0e3c422d81dc0dfe0da942b0"
format_stamp: "Formatted at 2026-06-21 07:56:41 on dist-test-slave-0pn3"
server_key: "4d547669f24b3d0007ce62d9b2e15400"
server_key_iv: "0174935fce21d26db014a47cab89ed6e"
server_key_version: "encryptionkey@0"
I20260621 07:56:41.575807 14558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:41.599206 14558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:41.601742 14558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:41.601927 14558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:41.608090 14558 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:43969
I20260621 07:56:41.608150 14625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:43969 every 8 connection(s)
I20260621 07:56:41.608983 14558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:41.611495 14626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:41.616390 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 14558
I20260621 07:56:41.616469 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:41.616726 12158 external_mini_cluster.cc:1428] Setting key 677e5c43d861172a2de448f398cb7e2a
I20260621 07:56:41.616745 14626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0: Bootstrap starting.
I20260621 07:56:41.619002 14626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:41.619827 14626 log.cc:824] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:41.621622 14626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0: No bootstrap required, opened a new log
Jun 21 07:56:41 dist-test-slave-0pn3 krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:41.625609 14626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43969 } }
I20260621 07:56:41.625813 14626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:41.625885 14626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06fd55eb0e3c422d81dc0dfe0da942b0, State: Initialized, Role: FOLLOWER
I20260621 07:56:41.626297 14626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43969 } }
I20260621 07:56:41.626463 14626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:41.626572 14626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:41.626708 14626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:41.627413 14626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43969 } }
I20260621 07:56:41.627732 14626 leader_election.cc:302] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06fd55eb0e3c422d81dc0dfe0da942b0; no voters:
I20260621 07:56:41.628041 14626 leader_election.cc:288] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:41.628098 14631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:41.628264 14631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [term 1 LEADER]: Becoming Leader. State: Replica: 06fd55eb0e3c422d81dc0dfe0da942b0, State: Running, Role: LEADER
I20260621 07:56:41.628638 14631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43969 } }
I20260621 07:56:41.629204 14626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:41.629704 14631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43969 } } }
I20260621 07:56:41.629896 14631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:41.630128 14632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06fd55eb0e3c422d81dc0dfe0da942b0. Latest consensus state: current_term: 1 leader_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fd55eb0e3c422d81dc0dfe0da942b0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43969 } } }
I20260621 07:56:41.630303 14632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:41.630364 14637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:41.630692 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:41.617865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46616 (local address 127.11.223.190:43969)
0621 07:56:41.618244 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:41.618258 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:41.618290 (+ 32us) server_negotiation.cc:407] Connection header received
0621 07:56:41.618900 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:41.618915 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:41.619261 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:41.619519 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:41.620440 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:41.621428 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:41.622580 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:41.622995 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:41.623111 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:41.625651 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:41.625655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:41.625666 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:41.625689 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:41.627300 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:41.627773 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:41.627779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:41.627784 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:41.627938 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:41.628243 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:41.628246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:41.628248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:41.628781 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:41.628976 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:41.629357 (+ 381us) server_negotiation.cc:299] Negotiation successful
0621 07:56:41.629674 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":131,"threads_started":1}
I20260621 07:56:41.633440 14637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:41.638296 14637 catalog_manager.cc:1347] Generated new cluster ID: b6cf76d21f044617ba4698ad6fca993d
I20260621 07:56:41.638360 14637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:41.650053 14637 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:41.651158 14637 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:41.657372 14637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06fd55eb0e3c422d81dc0dfe0da942b0: Generated new TSK 0
I20260621 07:56:41.658001 14637 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:41.690997 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43969
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:36593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:41.831660 14652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:41.831924 14652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:41.831996 14652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:41.836032 14652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:41.836107 14652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:41.836210 14652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:41.841202 14652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:36593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:41.842173 14652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:41.843146 14652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:41.852350 14660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:41.852506 14652 server_base.cc:1034] running on GCE node
W20260621 07:56:41.852370 14657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:41.852401 14658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:41.853058 14652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:41.853652 14652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:41.854830 14652 hybrid_clock.cc:648] HybridClock initialized: now 1782028601854802 us; error 40 us; skew 500 ppm
Jun 21 07:56:41 dist-test-slave-0pn3 krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:41.857600 14652 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:41.858402 14652 webserver.cc:460] Webserver started at http://127.11.223.129:37119/ using document root <none> and password file <none>
I20260621 07:56:41.858937 14652 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:41.859025 14652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:41.859246 14652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:41.861214 14652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "4b0f324f2cec451f80af8bf3686ae53f"
format_stamp: "Formatted at 2026-06-21 07:56:41 on dist-test-slave-0pn3"
server_key: "edf2774d693734ceb1ed1516345c1676"
server_key_iv: "04537fc4bd3498f5a99289fa607eaef6"
server_key_version: "encryptionkey@0"
I20260621 07:56:41.861838 14652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "4b0f324f2cec451f80af8bf3686ae53f"
format_stamp: "Formatted at 2026-06-21 07:56:41 on dist-test-slave-0pn3"
server_key: "edf2774d693734ceb1ed1516345c1676"
server_key_iv: "04537fc4bd3498f5a99289fa607eaef6"
server_key_version: "encryptionkey@0"
I20260621 07:56:41.865345 14652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260621 07:56:41.867686 14667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:41.868880 14652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:41.869006 14652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "4b0f324f2cec451f80af8bf3686ae53f"
format_stamp: "Formatted at 2026-06-21 07:56:41 on dist-test-slave-0pn3"
server_key: "edf2774d693734ceb1ed1516345c1676"
server_key_iv: "04537fc4bd3498f5a99289fa607eaef6"
server_key_version: "encryptionkey@0"
I20260621 07:56:41.869107 14652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:41.889415 14652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:41.891988 14652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:41.892120 14652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:41.892669 14652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:41.893388 14652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:41.893432 14652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:41.893468 14652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:41.893494 14652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:41.901827 14652 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:42097
I20260621 07:56:41.901870 14780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:42097 every 8 connection(s)
I20260621 07:56:41.902650 14652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:41.907006 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 14652
I20260621 07:56:41.907127 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:41.907378 12158 external_mini_cluster.cc:1428] Setting key c7d85d67431d1ee49bc73f3c1e763c5c
Jun 21 07:56:41 dist-test-slave-0pn3 krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:41.915462 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:41.904268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:56551 (local address 127.11.223.190:43969)
0621 07:56:41.904417 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:41.904420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:41.905125 (+ 705us) server_negotiation.cc:407] Connection header received
0621 07:56:41.906334 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:41.906337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:41.906392 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:41.906494 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:41.907679 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:41.908269 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:41.909445 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:41.909679 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:41.909782 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:41.912040 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:41.912045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:41.912052 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:41.912076 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:41.913502 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:41.913980 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:41.913985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:41.913987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:41.914094 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:41.914433 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:41.914438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:41.914440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:41.914612 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:41.914731 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:41.915185 (+ 454us) server_negotiation.cc:299] Negotiation successful
0621 07:56:41.915302 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:41.916214 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:41.904605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43969 (local address 127.11.223.129:56551)
0621 07:56:41.904985 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:41.905023 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:41.906134 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:41.906657 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:41.906664 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:41.906979 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:41.907545 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:41.907559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:41.908424 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:41.908427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:41.909313 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:41.909322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:41.909832 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:41.909836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:41.910005 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:41.910558 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:41.910576 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:41.911892 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:41.913625 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:41.913630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:41.913643 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:41.913881 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:41.914209 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:41.914212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:41.914214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:41.914343 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:41.914729 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:41.914736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:41.914955 (+ 219us) client_negotiation.cc:769] Sending connection context
0621 07:56:41.915132 (+ 177us) client_negotiation.cc:240] Negotiation successful
0621 07:56:41.915413 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":93,"threads_started":1}
I20260621 07:56:41.917456 14781 heartbeater.cc:347] Connected to a master server at 127.11.223.190:43969
I20260621 07:56:41.917682 14781 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:41.918130 14781 heartbeater.cc:511] Master 127.11.223.190:43969 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:56:41.919589 14590 ts_manager.cc:194] Registered new tserver with Master: 4b0f324f2cec451f80af8bf3686ae53f (127.11.223.129:42097)
I20260621 07:56:41.920892 14590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:56551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:41.932114 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43969
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:36593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:42.033193 14786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:42.033401 14786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:42.033439 14786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:42.037321 14786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:42.037366 14786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:42.037420 14786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:42.042217 14786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:36593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:42.043054 14786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:42.043912 14786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:42.052641 14792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:42.052637 14791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:42.052711 14786 server_base.cc:1034] running on GCE node
W20260621 07:56:42.052641 14794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:42.053167 14786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:42.053706 14786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:42.054860 14786 hybrid_clock.cc:648] HybridClock initialized: now 1782028602054847 us; error 24 us; skew 500 ppm
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:42.057479 14786 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:42.058252 14786 webserver.cc:460] Webserver started at http://127.11.223.130:38833/ using document root <none> and password file <none>
I20260621 07:56:42.058784 14786 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:42.058871 14786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:42.059088 14786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:42.060941 14786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "0d41f05571974b8cb115829b204d9c25"
format_stamp: "Formatted at 2026-06-21 07:56:42 on dist-test-slave-0pn3"
server_key: "cf181ce92bf8413b70ea7a5160f6d0b4"
server_key_iv: "56b8471124a24220e2ab7f96718aaa7e"
server_key_version: "encryptionkey@0"
I20260621 07:56:42.061456 14786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "0d41f05571974b8cb115829b204d9c25"
format_stamp: "Formatted at 2026-06-21 07:56:42 on dist-test-slave-0pn3"
server_key: "cf181ce92bf8413b70ea7a5160f6d0b4"
server_key_iv: "56b8471124a24220e2ab7f96718aaa7e"
server_key_version: "encryptionkey@0"
I20260621 07:56:42.064740 14786 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:42.066869 14801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:42.067930 14786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:42.068048 14786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "0d41f05571974b8cb115829b204d9c25"
format_stamp: "Formatted at 2026-06-21 07:56:42 on dist-test-slave-0pn3"
server_key: "cf181ce92bf8413b70ea7a5160f6d0b4"
server_key_iv: "56b8471124a24220e2ab7f96718aaa7e"
server_key_version: "encryptionkey@0"
I20260621 07:56:42.068147 14786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:42.096683 14786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:42.099656 14786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:42.099857 14786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:42.100397 14786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:42.101128 14786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:42.101195 14786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:42.101264 14786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:42.101305 14786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:42.109932 14786 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:38153
I20260621 07:56:42.109968 14914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:38153 every 8 connection(s)
I20260621 07:56:42.110746 14786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:42.117638 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 14786
I20260621 07:56:42.117744 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:42.118045 12158 external_mini_cluster.cc:1428] Setting key e53236c301d26b115ac0507b4adcfa9e
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:42.123884 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:42.112423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:43151 (local address 127.11.223.190:43969)
0621 07:56:42.112557 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:42.112561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:42.113326 (+ 765us) server_negotiation.cc:407] Connection header received
0621 07:56:42.114490 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:42.114496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:42.114560 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:42.114642 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:42.115782 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.116295 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.117371 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.117733 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.117838 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:42.120492 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:42.120499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:42.120505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:42.120537 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:42.121988 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:42.122473 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:42.122479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:42.122482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:42.122563 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:42.122887 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:42.122891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:42.122893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:42.123062 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:42.123164 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:42.123608 (+ 444us) server_negotiation.cc:299] Negotiation successful
0621 07:56:42.123726 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:56:42.124643 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:42.112781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43969 (local address 127.11.223.130:43151)
0621 07:56:42.113181 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:42.113220 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:42.114283 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:42.114805 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:42.114811 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:42.115219 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:42.115661 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:42.115673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.116435 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.116437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:42.117270 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:42.117278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.117913 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.117916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:42.118043 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:42.118820 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:42.118835 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:42.120286 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:42.122095 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:42.122099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:42.122108 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:42.122368 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:42.122666 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:42.122669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:42.122671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:42.122794 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:42.123172 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:42.123179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:42.123387 (+ 208us) client_negotiation.cc:769] Sending connection context
0621 07:56:42.123557 (+ 170us) client_negotiation.cc:240] Negotiation successful
0621 07:56:42.123840 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260621 07:56:42.125842 14915 heartbeater.cc:347] Connected to a master server at 127.11.223.190:43969
I20260621 07:56:42.126065 14915 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:42.126490 14915 heartbeater.cc:511] Master 127.11.223.190:43969 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:56:42.127494 14590 ts_manager.cc:194] Registered new tserver with Master: 0d41f05571974b8cb115829b204d9c25 (127.11.223.130:38153)
I20260621 07:56:42.128046 14590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:43151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:42.143204 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43969
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:36593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:42.246927 14920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:42.247145 14920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:42.247203 14920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:42.251468 14920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:42.251534 14920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:42.251627 14920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:42.256629 14920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:36593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:42.257525 14920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:42.258486 14920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:42.267622 14925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:42.267807 14920 server_base.cc:1034] running on GCE node
W20260621 07:56:42.267611 14928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:42.267611 14926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:42.268401 14920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:42.268981 14920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:42.270156 14920 hybrid_clock.cc:648] HybridClock initialized: now 1782028602270130 us; error 40 us; skew 500 ppm
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:42.272938 14920 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:42.273815 14920 webserver.cc:460] Webserver started at http://127.11.223.131:42183/ using document root <none> and password file <none>
I20260621 07:56:42.274331 14920 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:42.274390 14920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:42.274641 14920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:42.276589 14920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "f782e085d865417db69c693592a1b170"
format_stamp: "Formatted at 2026-06-21 07:56:42 on dist-test-slave-0pn3"
server_key: "2a2042422646c1164130541ac129dca1"
server_key_iv: "6b1f74f7546471f8693c6ed65f3737d9"
server_key_version: "encryptionkey@0"
I20260621 07:56:42.277117 14920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "f782e085d865417db69c693592a1b170"
format_stamp: "Formatted at 2026-06-21 07:56:42 on dist-test-slave-0pn3"
server_key: "2a2042422646c1164130541ac129dca1"
server_key_iv: "6b1f74f7546471f8693c6ed65f3737d9"
server_key_version: "encryptionkey@0"
I20260621 07:56:42.280696 14920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:42.282821 14935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:42.283919 14920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:42.284034 14920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "f782e085d865417db69c693592a1b170"
format_stamp: "Formatted at 2026-06-21 07:56:42 on dist-test-slave-0pn3"
server_key: "2a2042422646c1164130541ac129dca1"
server_key_iv: "6b1f74f7546471f8693c6ed65f3737d9"
server_key_version: "encryptionkey@0"
I20260621 07:56:42.284128 14920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:42.297991 14920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:42.301065 14920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:42.301255 14920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:42.301791 14920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:42.302536 14920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:42.302579 14920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:42.302645 14920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:42.302695 14920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:42.310853 14920 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:46283
I20260621 07:56:42.310874 15048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:46283 every 8 connection(s)
I20260621 07:56:42.311642 14920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:42.318300 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 14920
I20260621 07:56:42.318413 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:42.318732 12158 external_mini_cluster.cc:1428] Setting key 000a68680c6ceb3c6b1a7e30eb03f68b
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:42.324110 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:42.313207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:46703 (local address 127.11.223.190:43969)
0621 07:56:42.313327 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:42.313330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:42.314016 (+ 686us) server_negotiation.cc:407] Connection header received
0621 07:56:42.315325 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:42.315328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:42.315391 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:42.315500 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:42.316619 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.317105 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.318195 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.318430 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.318581 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:42.321089 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:42.321092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:42.321093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:42.321115 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:42.322521 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:42.322929 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:42.322931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:42.322932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:42.323015 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:42.323278 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:42.323280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:42.323281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:42.323434 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:42.323508 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:42.323884 (+ 376us) server_negotiation.cc:299] Negotiation successful
0621 07:56:42.323970 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260621 07:56:42.324824 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:42.313467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43969 (local address 127.11.223.131:46703)
0621 07:56:42.313851 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:42.313886 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:42.315105 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:42.315671 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:42.315678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:42.315982 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:42.316484 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:42.316500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.317200 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.317202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:42.318062 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:42.318071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:42.318606 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:42.318611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:42.318778 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:42.319352 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:42.319368 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:42.320958 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:42.322618 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:42.322621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:42.322631 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:42.322851 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:42.323090 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:42.323091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:42.323093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:42.323205 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:42.323510 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:42.323514 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:42.323703 (+ 189us) client_negotiation.cc:769] Sending connection context
0621 07:56:42.323840 (+ 137us) client_negotiation.cc:240] Negotiation successful
0621 07:56:42.324043 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":90,"threads_started":1}
I20260621 07:56:42.325764 15049 heartbeater.cc:347] Connected to a master server at 127.11.223.190:43969
I20260621 07:56:42.325946 15049 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:42.326314 15049 heartbeater.cc:511] Master 127.11.223.190:43969 requested a full tablet report, sending...
I20260621 07:56:42.327148 14590 ts_manager.cc:194] Registered new tserver with Master: f782e085d865417db69c693592a1b170 (127.11.223.131:46283)
I20260621 07:56:42.327600 14590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:46703
I20260621 07:56:42.332213 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260621 07:56:42.343906 14629 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:42.342629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.11.223.190:43969)
0621 07:56:42.342772 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:42.342790 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:42.342913 (+ 123us) server_negotiation.cc:407] Connection header received
0621 07:56:42.343034 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:42.343037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:42.343075 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:42.343154 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:42.343784 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46620: BlockingRecv error: recv got EOF from 127.0.0.1:46620 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:56:42.345052 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 14652
I20260621 07:56:42.350418 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 14786
I20260621 07:56:42.355281 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 14920
I20260621 07:56:42.360225 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 14558
2026-06-21T07:56:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2030 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[15061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[15061](info): set up 2 sockets
Jun 21 07:56:42 dist-test-slave-0pn3 krb5kdc[15061](info): commencing operation
krb5kdc: starting...
W20260621 07:56:43.360610 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:43 dist-test-slave-0pn3 krb5kdc[15061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:43.478444 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:35545
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:43.581195 15074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:43.581458 15074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:43.581528 15074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:43.585577 15074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:43.585651 15074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:43.585688 15074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:43.585724 15074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:43.585758 15074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:43.590803 15074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:35545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:43.591796 15074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:43.592767 15074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:43.600656 15080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:43.600651 15082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:43.600744 15074 server_base.cc:1034] running on GCE node
W20260621 07:56:43.600651 15079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:43.601402 15074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:43.602396 15074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:43.603580 15074 hybrid_clock.cc:648] HybridClock initialized: now 1782028603603544 us; error 48 us; skew 500 ppm
Jun 21 07:56:43 dist-test-slave-0pn3 krb5kdc[15061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:43.610639 15074 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:43.611457 15074 webserver.cc:460] Webserver started at http://127.11.223.190:43553/ using document root <none> and password file <none>
I20260621 07:56:43.611984 15074 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:43.612085 15074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:43.612370 15074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:43.614339 15074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "2f81bcc24cbe42c286c69179f293c566"
format_stamp: "Formatted at 2026-06-21 07:56:43 on dist-test-slave-0pn3"
server_key: "edea43665eed6b5ca8d63121e60534d6"
server_key_iv: "c97665ec965dd86efa13cbc586278f7d"
server_key_version: "encryptionkey@0"
I20260621 07:56:43.614885 15074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "2f81bcc24cbe42c286c69179f293c566"
format_stamp: "Formatted at 2026-06-21 07:56:43 on dist-test-slave-0pn3"
server_key: "edea43665eed6b5ca8d63121e60534d6"
server_key_iv: "c97665ec965dd86efa13cbc586278f7d"
server_key_version: "encryptionkey@0"
I20260621 07:56:43.618317 15074 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260621 07:56:43.620570 15089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:43.621626 15074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:56:43.621752 15074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "2f81bcc24cbe42c286c69179f293c566"
format_stamp: "Formatted at 2026-06-21 07:56:43 on dist-test-slave-0pn3"
server_key: "edea43665eed6b5ca8d63121e60534d6"
server_key_iv: "c97665ec965dd86efa13cbc586278f7d"
server_key_version: "encryptionkey@0"
I20260621 07:56:43.621886 15074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:43.644898 15074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:43.647439 15074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:43.647625 15074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:43.653928 15074 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:35545
I20260621 07:56:43.654201 15141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:35545 every 8 connection(s)
I20260621 07:56:43.654800 15074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:43.657263 15142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:43.662391 15142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566: Bootstrap starting.
I20260621 07:56:43.663903 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15074
I20260621 07:56:43.664089 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:43.664400 12158 external_mini_cluster.cc:1428] Setting key c7c0694c74c7417682fc1b0bcc2f1efc
I20260621 07:56:43.664839 15142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:43.665693 15142 log.cc:824] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:43.667514 15142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566: No bootstrap required, opened a new log
I20260621 07:56:43.671602 15142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f81bcc24cbe42c286c69179f293c566" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35545 } }
I20260621 07:56:43.671864 15142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:43.671921 15142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f81bcc24cbe42c286c69179f293c566, State: Initialized, Role: FOLLOWER
I20260621 07:56:43.672435 15142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f81bcc24cbe42c286c69179f293c566" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35545 } }
I20260621 07:56:43.672601 15142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:43.672681 15142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:43.672780 15142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 0 FOLLOWER]: Advancing to term 1
Jun 21 07:56:43 dist-test-slave-0pn3 krb5kdc[15061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:43.673666 15142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f81bcc24cbe42c286c69179f293c566" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35545 } }
I20260621 07:56:43.673976 15142 leader_election.cc:302] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f81bcc24cbe42c286c69179f293c566; no voters:
I20260621 07:56:43.674288 15142 leader_election.cc:288] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:43.674352 15147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:43.674556 15147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [term 1 LEADER]: Becoming Leader. State: Replica: 2f81bcc24cbe42c286c69179f293c566, State: Running, Role: LEADER
I20260621 07:56:43.674867 15147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f81bcc24cbe42c286c69179f293c566" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35545 } }
I20260621 07:56:43.675450 15142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:43.675982 15147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f81bcc24cbe42c286c69179f293c566. Latest consensus state: current_term: 1 leader_uuid: "2f81bcc24cbe42c286c69179f293c566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f81bcc24cbe42c286c69179f293c566" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35545 } } }
I20260621 07:56:43.676753 15147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:43.676635 15148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f81bcc24cbe42c286c69179f293c566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f81bcc24cbe42c286c69179f293c566" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35545 } } }
I20260621 07:56:43.677052 15148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:43.677861 15156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:43.679170 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:43.665476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52444 (local address 127.11.223.190:35545)
0621 07:56:43.665926 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:43.665943 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:43.665988 (+ 45us) server_negotiation.cc:407] Connection header received
0621 07:56:43.666738 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:43.666755 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:43.667062 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:43.667319 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:43.668223 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:43.669250 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:43.670474 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:43.670908 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:43.671035 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:43.673784 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:43.673790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:43.673808 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:43.673837 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:43.675692 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:43.676181 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:43.676186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:43.676192 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:43.676340 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:43.676667 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:43.676670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:43.676672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:43.677118 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:43.678092 (+ 974us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:43.678238 (+ 146us) server_negotiation.cc:299] Negotiation successful
0621 07:56:43.678470 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":143,"threads_started":1}
I20260621 07:56:43.680843 15156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:43.682142 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:56:43.685702 15156 catalog_manager.cc:1347] Generated new cluster ID: 7636755550414aa48fca6cfb9d26efca
I20260621 07:56:43.685762 15156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:43.693445 15156 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:43.694481 15156 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:43.702064 15156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f81bcc24cbe42c286c69179f293c566: Generated new TSK 0
I20260621 07:56:43.702627 15156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:56:43.711194 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:43.705946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.11.223.190:35545)
0621 07:56:43.706075 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:43.706080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:43.706236 (+ 156us) server_negotiation.cc:407] Connection header received
0621 07:56:43.706347 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:43.706350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:43.706397 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:43.706476 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:43.706919 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:43.707744 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:43.708965 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:43.709347 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:43.709443 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:43.710250 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:43.710253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:43.710256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:43.710786 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:43.710818 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:43.710837 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:43.710926 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:43.710995 (+ 69us) server_negotiation.cc:299] Negotiation successful
0621 07:56:43.711060 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 07:56:43.715586 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15074
2026-06-21T07:56:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1353 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:43 dist-test-slave-0pn3 krb5kdc[15175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:43 dist-test-slave-0pn3 krb5kdc[15175](info): set up 2 sockets
Jun 21 07:56:43 dist-test-slave-0pn3 krb5kdc[15175](info): commencing operation
krb5kdc: starting...
W20260621 07:56:44.716794 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:44 dist-test-slave-0pn3 krb5kdc[15175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:44.826422 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46299
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:46299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:44.929358 15188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:44.929585 15188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:44.929622 15188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:44.933506 15188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:44.933552 15188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:44.933570 15188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:44.933586 15188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:44.933600 15188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:44.938632 15188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:46299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:44.939505 15188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:44.940426 15188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:44.948307 15194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:44.948392 15196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:44.948369 15193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:44.948499 15188 server_base.cc:1034] running on GCE node
I20260621 07:56:44.949244 15188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:44.950197 15188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:44.951349 15188 hybrid_clock.cc:648] HybridClock initialized: now 1782028604951336 us; error 26 us; skew 500 ppm
Jun 21 07:56:44 dist-test-slave-0pn3 krb5kdc[15175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:44.953874 15188 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:44.954636 15188 webserver.cc:460] Webserver started at http://127.11.223.190:34937/ using document root <none> and password file <none>
I20260621 07:56:44.955127 15188 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:44.955179 15188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:44.955389 15188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:44.957302 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "23cc5aa8064c4d8c811b65eec349329a"
format_stamp: "Formatted at 2026-06-21 07:56:44 on dist-test-slave-0pn3"
server_key: "a43337d3b21a6fa309a212100c4501c7"
server_key_iv: "b4d9d64ae0f5a1e4812f121fb474dd8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:44.957818 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "23cc5aa8064c4d8c811b65eec349329a"
format_stamp: "Formatted at 2026-06-21 07:56:44 on dist-test-slave-0pn3"
server_key: "a43337d3b21a6fa309a212100c4501c7"
server_key_iv: "b4d9d64ae0f5a1e4812f121fb474dd8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:44.961284 15188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:44.963456 15203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:44.964618 15188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:44.964745 15188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "23cc5aa8064c4d8c811b65eec349329a"
format_stamp: "Formatted at 2026-06-21 07:56:44 on dist-test-slave-0pn3"
server_key: "a43337d3b21a6fa309a212100c4501c7"
server_key_iv: "b4d9d64ae0f5a1e4812f121fb474dd8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:44.964844 15188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:44.981222 15188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:44.983808 15188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:44.983990 15188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:44.990459 15188 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:46299
I20260621 07:56:44.990717 15255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:46299 every 8 connection(s)
I20260621 07:56:44.991398 15188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:44.993800 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:44.998731 15256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a: Bootstrap starting.
I20260621 07:56:45.000965 15256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:45.001497 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15188
I20260621 07:56:45.001598 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:45.001749 15256 log.cc:824] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:45.001922 12158 external_mini_cluster.cc:1428] Setting key 8e191df9983045892388383a266f2bed
I20260621 07:56:45.003793 15256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a: No bootstrap required, opened a new log
I20260621 07:56:45.007009 15256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cc5aa8064c4d8c811b65eec349329a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46299 } }
I20260621 07:56:45.007228 15256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:45.007287 15256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cc5aa8064c4d8c811b65eec349329a, State: Initialized, Role: FOLLOWER
I20260621 07:56:45.007684 15256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cc5aa8064c4d8c811b65eec349329a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46299 } }
I20260621 07:56:45.007795 15256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:45.007879 15256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:45.007975 15256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:45.009027 15256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cc5aa8064c4d8c811b65eec349329a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46299 } }
I20260621 07:56:45.009343 15256 leader_election.cc:302] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23cc5aa8064c4d8c811b65eec349329a; no voters:
I20260621 07:56:45.009675 15256 leader_election.cc:288] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:45.009723 15261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:45.010025 15261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [term 1 LEADER]: Becoming Leader. State: Replica: 23cc5aa8064c4d8c811b65eec349329a, State: Running, Role: LEADER
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:45.010344 15261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cc5aa8064c4d8c811b65eec349329a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46299 } }
I20260621 07:56:45.010963 15256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:45.011786 15262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23cc5aa8064c4d8c811b65eec349329a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cc5aa8064c4d8c811b65eec349329a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46299 } } }
I20260621 07:56:45.011914 15262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:45.012253 15263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23cc5aa8064c4d8c811b65eec349329a. Latest consensus state: current_term: 1 leader_uuid: "23cc5aa8064c4d8c811b65eec349329a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cc5aa8064c4d8c811b65eec349329a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46299 } } }
I20260621 07:56:45.012439 15263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a [sys.catalog]: This master's current role is: LEADER
W20260621 07:56:45.015995 15276 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:56:45.016067 15276 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:56:45.016171 15270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:45.016685 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.003035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.11.223.190:46299)
0621 07:56:45.003436 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:45.003449 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:45.003480 (+ 31us) server_negotiation.cc:407] Connection header received
0621 07:56:45.004165 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:45.004181 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:45.004533 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:45.004828 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:45.005607 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.006496 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.007342 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.007923 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.008076 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.010859 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:45.010865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:45.010879 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:45.010910 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:45.013167 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.013650 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.013655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.013660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.013788 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.014131 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.014134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.014136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.014617 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:45.014804 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:45.015583 (+ 779us) server_negotiation.cc:299] Negotiation successful
0621 07:56:45.015859 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":128,"threads_started":1}
I20260621 07:56:45.017032 15270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:45.022363 15270 catalog_manager.cc:1347] Generated new cluster ID: 606326f6dddd4a2896621d4aa81ee441
I20260621 07:56:45.022428 15270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:45.035355 15270 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:45.036409 15270 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:45.042749 15270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23cc5aa8064c4d8c811b65eec349329a: Generated new TSK 0
I20260621 07:56:45.043568 15270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:45.079500 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46299
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:45.211663 15282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:45.211982 15282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:45.212070 15282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:45.215924 15282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:45.215982 15282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:45.216086 15282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:45.220973 15282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:45.221984 15282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:45.222922 15282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:45.231711 15290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:45.231720 15287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:45.231767 15282 server_base.cc:1034] running on GCE node
W20260621 07:56:45.231720 15288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:45.232379 15282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:45.232975 15282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:45.234189 15282 hybrid_clock.cc:648] HybridClock initialized: now 1782028605234166 us; error 40 us; skew 500 ppm
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:45.237066 15282 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:45.237824 15282 webserver.cc:460] Webserver started at http://127.11.223.129:37129/ using document root <none> and password file <none>
I20260621 07:56:45.238343 15282 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:45.238428 15282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:45.238643 15282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:45.240648 15282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "689c94ba85764de0acefc7b4aac3957b"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "89e5372bdc1ba7b987e9be93c8c86e05"
server_key_iv: "1b56c308586bc4bf71f0d09495148a52"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.241178 15282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "689c94ba85764de0acefc7b4aac3957b"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "89e5372bdc1ba7b987e9be93c8c86e05"
server_key_iv: "1b56c308586bc4bf71f0d09495148a52"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.244608 15282 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:45.246838 15297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.247908 15282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:45.248041 15282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "689c94ba85764de0acefc7b4aac3957b"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "89e5372bdc1ba7b987e9be93c8c86e05"
server_key_iv: "1b56c308586bc4bf71f0d09495148a52"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.248143 15282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:45.279181 15282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:45.281965 15282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:45.282171 15282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:45.282729 15282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:45.283512 15282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:45.283583 15282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.283653 15282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:45.283694 15282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.292280 15282 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:45747
I20260621 07:56:45.292383 15410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:45747 every 8 connection(s)
I20260621 07:56:45.293339 15282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:45.297082 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15282
I20260621 07:56:45.297206 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:45.297478 12158 external_mini_cluster.cc:1428] Setting key a3cf1d01f6318d93adc394b9e2e2442f
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:45.307440 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.295233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:42717 (local address 127.11.223.190:46299)
0621 07:56:45.295386 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:45.295391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:45.296234 (+ 843us) server_negotiation.cc:407] Connection header received
0621 07:56:45.297787 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:45.297791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:45.297847 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:45.297969 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:45.299299 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.299818 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.301010 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.301222 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.301329 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.304025 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:45.304030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:45.304034 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:45.304062 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:45.305436 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.305936 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.305944 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.305945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.306018 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.306359 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.306363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.306365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.306542 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:45.306648 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:45.307114 (+ 466us) server_negotiation.cc:299] Negotiation successful
0621 07:56:45.307229 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:56:45.308252 15413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.295667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46299 (local address 127.11.223.129:42717)
0621 07:56:45.296082 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:45.296124 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:45.297584 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:45.298155 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:45.298162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:45.298642 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:45.299160 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:45.299174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.299944 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.299948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:45.300880 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:45.300889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.301364 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.301368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:45.301535 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.302156 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:45.302172 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:45.303865 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:45.305558 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:45.305563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:45.305576 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:45.305823 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:45.306108 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:45.306112 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:45.306114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:45.306278 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:45.306656 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:45.306663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:45.306887 (+ 224us) client_negotiation.cc:769] Sending connection context
0621 07:56:45.307077 (+ 190us) client_negotiation.cc:240] Negotiation successful
0621 07:56:45.307402 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:56:45.309504 15411 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46299
I20260621 07:56:45.309754 15411 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:45.310246 15411 heartbeater.cc:511] Master 127.11.223.190:46299 requested a full tablet report, sending...
I20260621 07:56:45.311724 15220 ts_manager.cc:194] Registered new tserver with Master: 689c94ba85764de0acefc7b4aac3957b (127.11.223.129:45747)
I20260621 07:56:45.312933 15220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:42717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:45.324162 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46299
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:45.431988 15416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:45.432209 15416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:45.432248 15416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:45.436141 15416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:45.436192 15416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:45.436244 15416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:45.441197 15416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:45.442066 15416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:45.442930 15416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:56:45.451722 15416 server_base.cc:1034] running on GCE node
W20260621 07:56:45.451738 15422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:45.451910 15421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:45.451749 15424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:45.452464 15416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:45.453019 15416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:45.454180 15416 hybrid_clock.cc:648] HybridClock initialized: now 1782028605454162 us; error 29 us; skew 500 ppm
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:45.456892 15416 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:45.457641 15416 webserver.cc:460] Webserver started at http://127.11.223.130:40457/ using document root <none> and password file <none>
I20260621 07:56:45.458117 15416 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:45.458184 15416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:45.458328 15416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:45.460188 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "1baf71fdd0ed4eb6b53a7021fe080309"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "01349c50d01dec6398cc87b09a508cb6"
server_key_iv: "20c21f3e0e655d3fae2b991895cae6d1"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.460760 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "1baf71fdd0ed4eb6b53a7021fe080309"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "01349c50d01dec6398cc87b09a508cb6"
server_key_iv: "20c21f3e0e655d3fae2b991895cae6d1"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.464069 15416 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:45.466286 15431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.467424 15416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:45.467540 15416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "1baf71fdd0ed4eb6b53a7021fe080309"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "01349c50d01dec6398cc87b09a508cb6"
server_key_iv: "20c21f3e0e655d3fae2b991895cae6d1"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.467638 15416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:45.487672 15416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:45.490842 15416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:45.491036 15416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:45.491591 15416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:45.492385 15416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:45.492431 15416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.492491 15416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:45.492527 15416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.501056 15416 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:43273
I20260621 07:56:45.501123 15544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:43273 every 8 connection(s)
I20260621 07:56:45.501926 15416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:45.510145 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15416
I20260621 07:56:45.510282 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:45.510584 12158 external_mini_cluster.cc:1428] Setting key 2b1eb67afa37c649b2e6ad9ab07aa69c
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:45.516404 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.503616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:52193 (local address 127.11.223.190:46299)
0621 07:56:45.503744 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:45.503747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:45.504485 (+ 738us) server_negotiation.cc:407] Connection header received
0621 07:56:45.505780 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:45.505783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:45.505837 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:45.505935 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:45.507086 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.507884 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.509192 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.509426 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.509535 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.512281 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:45.512285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:45.512287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:45.512341 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:45.514178 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.514773 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.514775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.514777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.514845 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.515240 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.515242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.515243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.515441 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:45.515548 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:45.516135 (+ 587us) server_negotiation.cc:299] Negotiation successful
0621 07:56:45.516241 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:56:45.517554 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.503898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46299 (local address 127.11.223.130:52193)
0621 07:56:45.504291 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:45.504384 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:45.505583 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:45.506095 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:45.506101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:45.506421 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:45.506875 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:45.506888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.508037 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.508043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:45.509068 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:45.509077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.509572 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.509577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:45.509747 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.510388 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:45.510411 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:45.512119 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:45.514292 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:45.514299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:45.514316 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:45.514669 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:45.514944 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:45.514965 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:45.514967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:45.515146 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:45.515561 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:45.515570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:45.515876 (+ 306us) client_negotiation.cc:769] Sending connection context
0621 07:56:45.516094 (+ 218us) client_negotiation.cc:240] Negotiation successful
0621 07:56:45.516481 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260621 07:56:45.518744 15545 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46299
I20260621 07:56:45.518987 15545 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:45.519431 15545 heartbeater.cc:511] Master 127.11.223.190:46299 requested a full tablet report, sending...
I20260621 07:56:45.520471 15220 ts_manager.cc:194] Registered new tserver with Master: 1baf71fdd0ed4eb6b53a7021fe080309 (127.11.223.130:43273)
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:56:45.521071 15220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:52193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:45.537333 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46299
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:45.642871 15550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:45.643157 15550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:45.643230 15550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:45.647037 15550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:45.647122 15550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:45.647204 15550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:45.652561 15550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:45.653523 15550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:45.654479 15550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:45.663704 15555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:45.663682 15558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:45.663678 15556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:45.663877 15550 server_base.cc:1034] running on GCE node
I20260621 07:56:45.664394 15550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:45.664984 15550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:45.666167 15550 hybrid_clock.cc:648] HybridClock initialized: now 1782028605666154 us; error 28 us; skew 500 ppm
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:45.668968 15550 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:45.669749 15550 webserver.cc:460] Webserver started at http://127.11.223.131:41811/ using document root <none> and password file <none>
I20260621 07:56:45.670284 15550 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:45.670372 15550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:45.670576 15550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:45.672646 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "e1206e8b9f0b45b3b9550ed97d721c71"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "b5282de935f243ba6329d87fcc0f80f2"
server_key_iv: "8d6cbc04aa4cc1b3d839c9d15186a91e"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.673170 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "e1206e8b9f0b45b3b9550ed97d721c71"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "b5282de935f243ba6329d87fcc0f80f2"
server_key_iv: "8d6cbc04aa4cc1b3d839c9d15186a91e"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.676594 15550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:45.678759 15565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.679930 15550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:45.680071 15550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "e1206e8b9f0b45b3b9550ed97d721c71"
format_stamp: "Formatted at 2026-06-21 07:56:45 on dist-test-slave-0pn3"
server_key: "b5282de935f243ba6329d87fcc0f80f2"
server_key_iv: "8d6cbc04aa4cc1b3d839c9d15186a91e"
server_key_version: "encryptionkey@0"
I20260621 07:56:45.680174 15550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:45.693465 15550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:45.696514 15550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:45.696693 15550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:45.697212 15550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:45.697986 15550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:45.698057 15550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.698127 15550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:45.698170 15550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:45.706470 15550 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:37697
I20260621 07:56:45.706504 15678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:37697 every 8 connection(s)
I20260621 07:56:45.707275 15550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:45.712721 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15550
I20260621 07:56:45.712893 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:45.713193 12158 external_mini_cluster.cc:1428] Setting key 9f0207c31fd869904903f255e625aad8
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:45.720556 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.708968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:39501 (local address 127.11.223.190:46299)
0621 07:56:45.709105 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:45.709108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:45.709853 (+ 745us) server_negotiation.cc:407] Connection header received
0621 07:56:45.711176 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:45.711180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:45.711236 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:45.711336 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:45.712770 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.713473 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.714934 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.715237 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.715352 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.717654 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:45.717657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:45.717659 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:45.717682 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:45.718866 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.719299 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.719301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.719302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.719375 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.719652 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.719653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.719655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.719811 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:45.719884 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:45.720294 (+ 410us) server_negotiation.cc:299] Negotiation successful
0621 07:56:45.720438 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 07:56:45.721338 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.709317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46299 (local address 127.11.223.131:39501)
0621 07:56:45.709714 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:45.709751 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:45.710962 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:45.711504 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:45.711511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:45.711911 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:45.712594 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:45.712618 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.713608 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.713611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:45.714818 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:45.714827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.715384 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.715388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:45.715565 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.716041 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:45.716056 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:45.717518 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:45.718964 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:45.718968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:45.718979 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:45.719214 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:45.719450 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:45.719452 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:45.719453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:45.719577 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:45.719893 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:45.719897 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:45.720109 (+ 212us) client_negotiation.cc:769] Sending connection context
0621 07:56:45.720251 (+ 142us) client_negotiation.cc:240] Negotiation successful
0621 07:56:45.720549 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":104,"threads_started":1}
I20260621 07:56:45.722518 15679 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46299
I20260621 07:56:45.722709 15679 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:45.723110 15679 heartbeater.cc:511] Master 127.11.223.190:46299 requested a full tablet report, sending...
I20260621 07:56:45.723999 15220 ts_manager.cc:194] Registered new tserver with Master: e1206e8b9f0b45b3b9550ed97d721c71 (127.11.223.131:37697)
I20260621 07:56:45.724627 15220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:39501
I20260621 07:56:45.726778 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028605, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:45.750211 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:45.742713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.11.223.190:46299)
0621 07:56:45.742853 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:45.742858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:45.743020 (+ 162us) server_negotiation.cc:407] Connection header received
0621 07:56:45.743120 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:45.743122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:45.743163 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:45.743231 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:45.743833 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.744369 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.745107 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:45.745299 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:45.745367 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:45.747544 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:45.747546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:45.747548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:45.747568 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:45.748787 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.749162 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.749164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.749165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.749232 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:45.749445 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:45.749447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:45.749448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:45.749638 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:45.749735 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:45.749916 (+ 181us) server_negotiation.cc:299] Negotiation successful
0621 07:56:45.750006 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260621 07:56:45.750559 15220 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40074
I20260621 07:56:45.751325 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15282
I20260621 07:56:45.756364 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15416
I20260621 07:56:45.761415 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15550
I20260621 07:56:45.766433 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15188
2026-06-21T07:56:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2052 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15691](info): set up 2 sockets
Jun 21 07:56:45 dist-test-slave-0pn3 krb5kdc[15691](info): commencing operation
krb5kdc: starting...
W20260621 07:56:46.788879 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:46 dist-test-slave-0pn3 krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:46.898859 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:36105
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:36105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:47.000519 15704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:47.000774 15704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:47.000842 15704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:47.004822 15704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:47.004900 15704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:47.004937 15704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:47.004995 15704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:47.005035 15704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:47.010073 15704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:36105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:36105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:47.011080 15704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:47.012072 15704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:47.020334 15712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.020367 15710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.020414 15709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:47.020605 15704 server_base.cc:1034] running on GCE node
I20260621 07:56:47.021131 15704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:47.022120 15704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:47.023299 15704 hybrid_clock.cc:648] HybridClock initialized: now 1782028607023291 us; error 27 us; skew 500 ppm
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:47.026345 15704 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:47.027153 15704 webserver.cc:460] Webserver started at http://127.11.223.190:41451/ using document root <none> and password file <none>
I20260621 07:56:47.027606 15704 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:47.027657 15704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:47.027796 15704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:47.029618 15704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "433b18b89ffc49b196a0dd3504861d24"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "10fea3585c3aa758fed04918b767a9eb"
server_key_iv: "7e3bfa4f6ca6b0dc24cc240367421496"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.030118 15704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "433b18b89ffc49b196a0dd3504861d24"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "10fea3585c3aa758fed04918b767a9eb"
server_key_iv: "7e3bfa4f6ca6b0dc24cc240367421496"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.033540 15704 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:47.035679 15719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.036907 15704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:47.037037 15704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "433b18b89ffc49b196a0dd3504861d24"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "10fea3585c3aa758fed04918b767a9eb"
server_key_iv: "7e3bfa4f6ca6b0dc24cc240367421496"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.037129 15704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:47.047762 15704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:47.050705 15704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:47.050895 15704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:47.057207 15704 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:36105
I20260621 07:56:47.057279 15771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:36105 every 8 connection(s)
I20260621 07:56:47.058065 15704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:47.060621 15772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:47.063493 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15704
I20260621 07:56:47.063634 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:47.063902 12158 external_mini_cluster.cc:1428] Setting key 3ad4897276108d72d4fa63329d4d83c1
I20260621 07:56:47.065933 15772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24: Bootstrap starting.
I20260621 07:56:47.068291 15772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:47.069137 15772 log.cc:824] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:47.071130 15772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24: No bootstrap required, opened a new log
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:47.074132 15772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433b18b89ffc49b196a0dd3504861d24" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36105 } }
I20260621 07:56:47.074348 15772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:47.074440 15772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433b18b89ffc49b196a0dd3504861d24, State: Initialized, Role: FOLLOWER
I20260621 07:56:47.074843 15772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433b18b89ffc49b196a0dd3504861d24" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36105 } }
I20260621 07:56:47.074954 15772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:47.075024 15772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:47.075115 15772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:47.075855 15772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433b18b89ffc49b196a0dd3504861d24" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36105 } }
I20260621 07:56:47.076165 15772 leader_election.cc:302] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 433b18b89ffc49b196a0dd3504861d24; no voters:
I20260621 07:56:47.076510 15772 leader_election.cc:288] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:47.076567 15777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:47.076735 15777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [term 1 LEADER]: Becoming Leader. State: Replica: 433b18b89ffc49b196a0dd3504861d24, State: Running, Role: LEADER
I20260621 07:56:47.077049 15777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433b18b89ffc49b196a0dd3504861d24" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36105 } }
I20260621 07:56:47.077623 15772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:47.078552 15778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433b18b89ffc49b196a0dd3504861d24. Latest consensus state: current_term: 1 leader_uuid: "433b18b89ffc49b196a0dd3504861d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433b18b89ffc49b196a0dd3504861d24" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36105 } } }
I20260621 07:56:47.078913 15778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:47.079089 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.065163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39134 (local address 127.11.223.190:36105)
0621 07:56:47.065517 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:47.065531 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:47.065562 (+ 31us) server_negotiation.cc:407] Connection header received
0621 07:56:47.066164 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:47.066180 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:47.066480 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:47.066729 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:47.067624 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.068720 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.069807 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.070264 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.070391 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.073051 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:47.073055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:47.073066 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:47.073090 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:47.075508 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.075946 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.075953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.075962 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.076115 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.076425 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.076429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.076431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.076934 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:47.077636 (+ 702us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:47.077788 (+ 152us) server_negotiation.cc:299] Negotiation successful
0621 07:56:47.078066 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":122,"threads_started":1}
I20260621 07:56:47.078824 15779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "433b18b89ffc49b196a0dd3504861d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433b18b89ffc49b196a0dd3504861d24" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36105 } } }
I20260621 07:56:47.079344 15779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:47.079911 15787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:47.082963 15787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:47.088057 15787 catalog_manager.cc:1347] Generated new cluster ID: f3960d075682411284947fb79faaf17a
I20260621 07:56:47.088116 15787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:47.104386 15787 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:47.105494 15787 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:47.110069 15787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 433b18b89ffc49b196a0dd3504861d24: Generated new TSK 0
I20260621 07:56:47.110651 15787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:47.141508 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36105
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:47.246446 15798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:47.246696 15798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:47.246770 15798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:47.250679 15798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:47.250757 15798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:47.250841 15798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:47.255775 15798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:47.256690 15798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:47.257596 15798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:47.266240 15806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.266234 15803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:47.266405 15798 server_base.cc:1034] running on GCE node
W20260621 07:56:47.266616 15804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:47.267045 15798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:47.267557 15798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:47.268705 15798 hybrid_clock.cc:648] HybridClock initialized: now 1782028607268693 us; error 25 us; skew 500 ppm
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:47.271447 15798 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:47.272305 15798 webserver.cc:460] Webserver started at http://127.11.223.129:36817/ using document root <none> and password file <none>
I20260621 07:56:47.272873 15798 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:47.272931 15798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:47.273172 15798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:47.275079 15798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "6e73d190a06c4027ba6b8d8eb111d7df"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "33b11837974badcf35ad15176c3bec3a"
server_key_iv: "6bea7f8b3aede13b561db540db7393b2"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.275650 15798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "6e73d190a06c4027ba6b8d8eb111d7df"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "33b11837974badcf35ad15176c3bec3a"
server_key_iv: "6bea7f8b3aede13b561db540db7393b2"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.279713 15798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260621 07:56:47.281961 15813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.283061 15798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:47.283174 15798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "6e73d190a06c4027ba6b8d8eb111d7df"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "33b11837974badcf35ad15176c3bec3a"
server_key_iv: "6bea7f8b3aede13b561db540db7393b2"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.283269 15798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:47.307093 15798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:47.310366 15798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:47.310571 15798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:47.311136 15798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:47.311895 15798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:47.311940 15798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.312018 15798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:47.312052 15798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.321048 15798 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:39621
I20260621 07:56:47.321075 15926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:39621 every 8 connection(s)
I20260621 07:56:47.321825 15798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:47.327086 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15798
I20260621 07:56:47.327214 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:47.327497 12158 external_mini_cluster.cc:1428] Setting key 199b321dbd6187e51f873f3d4611c610
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:47.334707 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.323388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:59051 (local address 127.11.223.190:36105)
0621 07:56:47.323520 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:47.323523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:47.324281 (+ 758us) server_negotiation.cc:407] Connection header received
0621 07:56:47.325558 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:47.325560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:47.325602 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:47.325669 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:47.326749 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.327308 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.328669 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.328893 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.329029 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.331291 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:47.331294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:47.331297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:47.331327 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:47.332782 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.333268 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.333272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.333274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.333355 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.333686 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.333689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.333691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.333856 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:47.333986 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:47.334435 (+ 449us) server_negotiation.cc:299] Negotiation successful
0621 07:56:47.334547 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:47.335518 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.323731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:36105 (local address 127.11.223.129:59051)
0621 07:56:47.324139 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:47.324177 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:47.325382 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:47.325807 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:47.325813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:47.326120 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:47.326633 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:47.326645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.327438 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.327442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:47.328570 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:47.328582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.329061 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.329065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:47.329228 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.329770 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:47.329785 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:47.331140 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:47.332908 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:47.332913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:47.332926 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:47.333167 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:47.333448 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:47.333451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:47.333453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:47.333587 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:47.333977 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:47.333982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:47.334207 (+ 225us) client_negotiation.cc:769] Sending connection context
0621 07:56:47.334398 (+ 191us) client_negotiation.cc:240] Negotiation successful
0621 07:56:47.334685 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:47.336716 15927 heartbeater.cc:347] Connected to a master server at 127.11.223.190:36105
I20260621 07:56:47.336984 15927 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:47.337458 15927 heartbeater.cc:511] Master 127.11.223.190:36105 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:56:47.339135 15736 ts_manager.cc:194] Registered new tserver with Master: 6e73d190a06c4027ba6b8d8eb111d7df (127.11.223.129:39621)
I20260621 07:56:47.340535 15736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:59051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:47.352896 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36105
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:47.455533 15932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:47.455809 15932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:47.455881 15932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:47.459755 15932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:47.459843 15932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:47.459933 15932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:47.464970 15932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:47.465919 15932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:47.466893 15932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:47.475642 15937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.475617 15940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.475658 15938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:47.475816 15932 server_base.cc:1034] running on GCE node
I20260621 07:56:47.476289 15932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:47.476928 15932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:47.478111 15932 hybrid_clock.cc:648] HybridClock initialized: now 1782028607478099 us; error 24 us; skew 500 ppm
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:47.480861 15932 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:47.481601 15932 webserver.cc:460] Webserver started at http://127.11.223.130:43633/ using document root <none> and password file <none>
I20260621 07:56:47.482081 15932 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:47.482163 15932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:47.482366 15932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:47.484164 15932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "c45d8071515c4482b676131a57be363d"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "80d4774545ec7c6c570fefb69b3f8211"
server_key_iv: "ded9ddcd542b21afd13590120242f6e0"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.484735 15932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "c45d8071515c4482b676131a57be363d"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "80d4774545ec7c6c570fefb69b3f8211"
server_key_iv: "ded9ddcd542b21afd13590120242f6e0"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.488154 15932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:47.490339 15947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.491425 15932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:47.491556 15932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "c45d8071515c4482b676131a57be363d"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "80d4774545ec7c6c570fefb69b3f8211"
server_key_iv: "ded9ddcd542b21afd13590120242f6e0"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.491657 15932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:47.518405 15932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:47.521472 15932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:47.521659 15932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:47.522176 15932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:47.522922 15932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:47.522987 15932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.523059 15932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:47.523098 15932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.531287 15932 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:34067
I20260621 07:56:47.531313 16060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:34067 every 8 connection(s)
I20260621 07:56:47.532094 15932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:47.538427 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 15932
I20260621 07:56:47.538575 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:47.538885 12158 external_mini_cluster.cc:1428] Setting key aafe5d6f6fc656467d25c59cb115a83b
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:47.545500 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.533838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:58311 (local address 127.11.223.190:36105)
0621 07:56:47.533975 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:47.533979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:47.534667 (+ 688us) server_negotiation.cc:407] Connection header received
0621 07:56:47.535896 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:47.535898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:47.535944 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:47.536023 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:47.537149 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.537675 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.538823 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.539156 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.539272 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.541898 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:47.541902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:47.541904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:47.541936 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:47.543408 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.544010 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.544012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.544013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.544086 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.544476 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.544480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.544482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.544658 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:47.544764 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:47.545224 (+ 460us) server_negotiation.cc:299] Negotiation successful
0621 07:56:47.545338 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:56:47.546211 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.534126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:36105 (local address 127.11.223.130:58311)
0621 07:56:47.534522 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:47.534560 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:47.535715 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:47.536173 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:47.536179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:47.536569 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:47.537029 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:47.537042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.537776 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.537778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:47.538699 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:47.538709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.539323 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.539328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:47.539517 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.540114 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:47.540141 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:47.541721 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:47.543528 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:47.543537 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:47.543552 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:47.543909 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:47.544185 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:47.544188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:47.544190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:47.544370 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:47.544762 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:47.544768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:47.544995 (+ 227us) client_negotiation.cc:769] Sending connection context
0621 07:56:47.545167 (+ 172us) client_negotiation.cc:240] Negotiation successful
0621 07:56:47.545471 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":104,"threads_started":1}
I20260621 07:56:47.547252 16061 heartbeater.cc:347] Connected to a master server at 127.11.223.190:36105
I20260621 07:56:47.547497 16061 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:56:47.547988 16061 heartbeater.cc:511] Master 127.11.223.190:36105 requested a full tablet report, sending...
I20260621 07:56:47.549006 15736 ts_manager.cc:194] Registered new tserver with Master: c45d8071515c4482b676131a57be363d (127.11.223.130:34067)
I20260621 07:56:47.549654 15736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:58311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:47.563242 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36105
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:47.667809 16066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:47.668025 16066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:47.668063 16066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:47.671940 16066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:47.671985 16066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:47.672037 16066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:47.676898 16066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:47.677734 16066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:47.678635 16066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:47.687373 16072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.687363 16074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:47.687454 16071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:47.687601 16066 server_base.cc:1034] running on GCE node
I20260621 07:56:47.688045 16066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:47.688669 16066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:47.689862 16066 hybrid_clock.cc:648] HybridClock initialized: now 1782028607689849 us; error 28 us; skew 500 ppm
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:47.692416 16066 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:47.693186 16066 webserver.cc:460] Webserver started at http://127.11.223.131:42123/ using document root <none> and password file <none>
I20260621 07:56:47.693729 16066 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:47.693817 16066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:47.694037 16066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:47.695932 16066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "b4d3a240057d4666b346d101ceed740b"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "7737b9c1c9246d9c905c545b44c3c396"
server_key_iv: "8d4134697b73306449e26c32b22fae3f"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.696476 16066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "b4d3a240057d4666b346d101ceed740b"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "7737b9c1c9246d9c905c545b44c3c396"
server_key_iv: "8d4134697b73306449e26c32b22fae3f"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.699771 16066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:47.702031 16081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.703152 16066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260621 07:56:47.703290 16066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "b4d3a240057d4666b346d101ceed740b"
format_stamp: "Formatted at 2026-06-21 07:56:47 on dist-test-slave-0pn3"
server_key: "7737b9c1c9246d9c905c545b44c3c396"
server_key_iv: "8d4134697b73306449e26c32b22fae3f"
server_key_version: "encryptionkey@0"
I20260621 07:56:47.703393 16066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:47.718879 16066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:47.721485 16066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:47.721663 16066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:47.722163 16066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:47.722863 16066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:47.722931 16066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.723003 16066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:47.723038 16066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:47.731233 16066 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:42791
I20260621 07:56:47.731257 16194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:42791 every 8 connection(s)
I20260621 07:56:47.731986 16066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:47.738590 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16066
I20260621 07:56:47.738703 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:47.738963 12158 external_mini_cluster.cc:1428] Setting key 5d1d93ebe30e47b6ba767e716ee9e9bc
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:47.744655 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.733703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:47447 (local address 127.11.223.190:36105)
0621 07:56:47.733842 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:47.733845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:47.734565 (+ 720us) server_negotiation.cc:407] Connection header received
0621 07:56:47.735888 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:47.735892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:47.735948 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:47.736055 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:47.737131 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.737656 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.738714 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.738997 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.739106 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.741684 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:47.741686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:47.741688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:47.741709 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:47.743066 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.743470 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.743472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.743473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.743543 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.743802 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.743804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.743805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.743955 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:47.744025 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:47.744470 (+ 445us) server_negotiation.cc:299] Negotiation successful
0621 07:56:47.744554 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:56:47.745352 16197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.734030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:36105 (local address 127.11.223.131:47447)
0621 07:56:47.734424 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:47.734462 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:47.735683 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:47.736207 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:47.736213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:47.736538 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:47.736983 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:47.736995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.737774 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.737779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:47.738592 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:47.738600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.739150 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.739154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:47.739307 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.739898 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:47.739922 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:47.741552 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:47.743154 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:47.743158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:47.743167 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:47.743384 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:47.743615 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:47.743617 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:47.743618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:47.743730 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:47.744036 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:47.744040 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:47.744224 (+ 184us) client_negotiation.cc:769] Sending connection context
0621 07:56:47.744442 (+ 218us) client_negotiation.cc:240] Negotiation successful
0621 07:56:47.744643 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260621 07:56:47.746335 16195 heartbeater.cc:347] Connected to a master server at 127.11.223.190:36105
I20260621 07:56:47.746515 16195 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:47.746872 16195 heartbeater.cc:511] Master 127.11.223.190:36105 requested a full tablet report, sending...
I20260621 07:56:47.747733 15736 ts_manager.cc:194] Registered new tserver with Master: b4d3a240057d4666b346d101ceed740b (127.11.223.131:42791)
I20260621 07:56:47.748188 15736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:47447
I20260621 07:56:47.752592 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:47.775580 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.767441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.11.223.129:39621)
0621 07:56:47.767749 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:47.767758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:47.767778 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:56:47.767832 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:47.767836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:47.767976 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:47.768080 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:47.768736 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.769483 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.770249 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.770581 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.770659 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.772895 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:47.772902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:47.772911 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:47.772932 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:47.774186 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.774522 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.774526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.774527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.774593 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.774792 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.774794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.774795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.774973 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:47.775097 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:47.775259 (+ 162us) server_negotiation.cc:299] Negotiation successful
0621 07:56:47.775404 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[15691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:47.785118 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:47.777999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.11.223.190:36105)
0621 07:56:47.778147 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:47.778150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:47.778161 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:47.778222 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:47.778224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:47.778256 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:47.778312 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:47.778899 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.779459 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.780170 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:47.780420 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:47.780499 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:47.782409 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:47.782411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:47.782413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:47.782432 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:47.783566 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.783956 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.783960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.783965 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.784044 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:47.784343 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:47.784346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:47.784348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:47.784541 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:47.784673 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:47.784881 (+ 208us) server_negotiation.cc:299] Negotiation successful
0621 07:56:47.784984 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260621 07:56:47.785447 15736 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39138
I20260621 07:56:47.786016 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15798
I20260621 07:56:47.792824 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15932
I20260621 07:56:47.797530 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16066
I20260621 07:56:47.802673 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 15704
2026-06-21T07:56:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2035 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[16207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[16207](info): set up 2 sockets
Jun 21 07:56:47 dist-test-slave-0pn3 krb5kdc[16207](info): commencing operation
krb5kdc: starting...
W20260621 07:56:48.808806 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:48 dist-test-slave-0pn3 krb5kdc[16207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:48.916561 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33995
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:33995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:49.016520 16220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:49.016798 16220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:49.016875 16220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:49.020730 16220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:49.020804 16220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:49.020841 16220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:49.020898 16220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:49.020942 16220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:49.025883 16220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:33995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:49.026823 16220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:49.027758 16220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:49.035501 16228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.035506 16225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.035506 16226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:49.035552 16220 server_base.cc:1034] running on GCE node
I20260621 07:56:49.036161 16220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:49.037161 16220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:49.038337 16220 hybrid_clock.cc:648] HybridClock initialized: now 1782028609038317 us; error 31 us; skew 500 ppm
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:49.046100 16220 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:49.046729 16220 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:49.046815 16220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:49.047020 16220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:49.048890 16220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "02ec7f8c1a1445d2be272d358ae4ac68"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "cd2baff966b734810bdd4091e0ebabb4"
server_key_iv: "96e61eb3722eefd63f6872b1f8c81a69"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.049443 16220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "02ec7f8c1a1445d2be272d358ae4ac68"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "cd2baff966b734810bdd4091e0ebabb4"
server_key_iv: "96e61eb3722eefd63f6872b1f8c81a69"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.052870 16220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:49.054989 16234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.056061 16220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:49.056183 16220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "02ec7f8c1a1445d2be272d358ae4ac68"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "cd2baff966b734810bdd4091e0ebabb4"
server_key_iv: "96e61eb3722eefd63f6872b1f8c81a69"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.056280 16220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:49.071398 16220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:49.074359 16220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:49.074529 16220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:49.080231 16220 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:33995
I20260621 07:56:49.080489 16286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:33995 every 8 connection(s)
I20260621 07:56:49.080802 16220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:49.081207 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16220
I20260621 07:56:49.081346 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:49.081593 12158 external_mini_cluster.cc:1428] Setting key e70185d34c9d1eab21f76abbcac1819e
I20260621 07:56:49.083643 16287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:49.089459 16287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68: Bootstrap starting.
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:49.092208 16287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:49.093151 16287 log.cc:824] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:49.095381 16287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68: No bootstrap required, opened a new log
I20260621 07:56:49.096273 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.082770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37264 (local address 127.11.223.190:33995)
0621 07:56:49.083132 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.083148 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.083183 (+ 35us) server_negotiation.cc:407] Connection header received
0621 07:56:49.083991 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.084011 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.084393 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.084719 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:49.085347 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.086640 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.087391 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.088036 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.088185 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.090649 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:49.090653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:49.090683 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:49.090720 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:49.092828 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.093345 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.093350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.093355 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.093504 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.093904 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.093909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.093912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.094359 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:49.094529 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.095210 (+ 681us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.095453 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260621 07:56:49.099684 16287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33995 } }
I20260621 07:56:49.099900 16287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:49.099964 16287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ec7f8c1a1445d2be272d358ae4ac68, State: Initialized, Role: FOLLOWER
I20260621 07:56:49.100427 16287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33995 } }
I20260621 07:56:49.100589 16287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:49.100649 16287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:49.100739 16287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:49.101537 16287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33995 } }
I20260621 07:56:49.101863 16287 leader_election.cc:302] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02ec7f8c1a1445d2be272d358ae4ac68; no voters:
I20260621 07:56:49.102138 16287 leader_election.cc:288] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:49.102270 16292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:49.102507 16292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [term 1 LEADER]: Becoming Leader. State: Replica: 02ec7f8c1a1445d2be272d358ae4ac68, State: Running, Role: LEADER
I20260621 07:56:49.102818 16292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33995 } }
I20260621 07:56:49.103312 16287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:49.104514 16293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33995 } } }
I20260621 07:56:49.104637 16293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:49.104818 16294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ec7f8c1a1445d2be272d358ae4ac68. Latest consensus state: current_term: 1 leader_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ec7f8c1a1445d2be272d358ae4ac68" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33995 } } }
I20260621 07:56:49.104897 16294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:49.105850 16301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:49.108148 16301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:49.112898 16301 catalog_manager.cc:1347] Generated new cluster ID: e600c568e8b04e8eb7529fc937a19662
I20260621 07:56:49.112954 16301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:49.137698 16301 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:49.138713 16301 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:49.152544 16301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68: Generated new TSK 0
I20260621 07:56:49.153259 16301 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:49.187288 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33995
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:49.290805 16313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:49.291026 16313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:49.291065 16313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:49.294934 16313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:49.294981 16313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:49.295053 16313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:49.299863 16313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:49.300706 16313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:49.301523 16313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:49.310241 16321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.310248 16319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.310252 16318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:49.310693 16313 server_base.cc:1034] running on GCE node
I20260621 07:56:49.311080 16313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:49.311653 16313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:49.312849 16313 hybrid_clock.cc:648] HybridClock initialized: now 1782028609312836 us; error 26 us; skew 500 ppm
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:49.315380 16313 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:49.316013 16313 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:49.316102 16313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:49.316347 16313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:49.318157 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "209fec5a2b174b1aadd400988d784a1d"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "703db6f3d37fd659221d14a3389b59ab"
server_key_iv: "231ebb6ab17f11d2ec5825af50f3997e"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.318688 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "209fec5a2b174b1aadd400988d784a1d"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "703db6f3d37fd659221d14a3389b59ab"
server_key_iv: "231ebb6ab17f11d2ec5825af50f3997e"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.322059 16313 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:49.324226 16327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.325325 16313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:49.325489 16313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "209fec5a2b174b1aadd400988d784a1d"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "703db6f3d37fd659221d14a3389b59ab"
server_key_iv: "231ebb6ab17f11d2ec5825af50f3997e"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.325599 16313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:49.343847 16313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:49.346292 16313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:49.346431 16313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:49.346868 16313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:49.347514 16313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:49.347556 16313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.347589 16313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:49.347609 16313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.355633 16313 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:35989
I20260621 07:56:49.355652 16440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:35989 every 8 connection(s)
I20260621 07:56:49.356104 16313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:49.362641 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16313
I20260621 07:56:49.362766 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:49.363047 12158 external_mini_cluster.cc:1428] Setting key 5a179cd9f955fc7308373e8912b17381
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:49.368664 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.357605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:51839 (local address 127.11.223.190:33995)
0621 07:56:49.357738 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.357742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.358460 (+ 718us) server_negotiation.cc:407] Connection header received
0621 07:56:49.359734 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.359737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.359794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.359909 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:49.360988 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.361463 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.362440 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.362633 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.362699 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.365109 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:49.365112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:49.365114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:49.365135 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:49.366716 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.367266 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.367268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.367269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.367336 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.367670 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.367671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.367672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.367810 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:49.367877 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.368456 (+ 579us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.368561 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:49.369469 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.357911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33995 (local address 127.11.223.129:51839)
0621 07:56:49.358335 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.358370 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.359529 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.360077 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.360084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.360477 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:49.360872 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.360883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.361556 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.361558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.362326 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.362346 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.362772 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.362778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.362954 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.363469 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:49.363482 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:49.364961 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:49.366821 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:49.366825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:49.366834 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:49.367166 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:49.367415 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:49.367419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:49.367420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:49.367584 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:49.367901 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:49.367909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:49.368183 (+ 274us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.368412 (+ 229us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.368719 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":115,"threads_started":1}
I20260621 07:56:49.370519 16441 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33995
I20260621 07:56:49.370740 16441 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:49.371189 16441 heartbeater.cc:511] Master 127.11.223.190:33995 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:49.372613 16251 ts_manager.cc:194] Registered new tserver with Master: 209fec5a2b174b1aadd400988d784a1d (127.11.223.129:35989)
I20260621 07:56:49.373836 16251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:51839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:49.388600 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33995
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:49.503568 16446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:49.503830 16446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:49.503912 16446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:49.510421 16446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:49.510506 16446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:49.510604 16446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:49.517133 16446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:49.518051 16446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:49.518970 16446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:49.527563 16452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.527549 16451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.527573 16454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:49.527849 16446 server_base.cc:1034] running on GCE node
I20260621 07:56:49.528270 16446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:49.528849 16446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:49.530032 16446 hybrid_clock.cc:648] HybridClock initialized: now 1782028609530007 us; error 44 us; skew 500 ppm
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:49.532711 16446 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:49.533351 16446 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:49.533442 16446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:49.533658 16446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:49.535516 16446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "cd8b2f48cc954c6caaff09d52abfa87b"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "d4684fbbb9ee942b04d8290c9f5e844a"
server_key_iv: "1e590d0ee571e1a1aa4da8f1121a8854"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.536059 16446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "cd8b2f48cc954c6caaff09d52abfa87b"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "d4684fbbb9ee942b04d8290c9f5e844a"
server_key_iv: "1e590d0ee571e1a1aa4da8f1121a8854"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.539433 16446 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:49.541683 16460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.542760 16446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:49.542930 16446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "cd8b2f48cc954c6caaff09d52abfa87b"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "d4684fbbb9ee942b04d8290c9f5e844a"
server_key_iv: "1e590d0ee571e1a1aa4da8f1121a8854"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.543035 16446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:49.567956 16446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:49.570614 16446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:49.570806 16446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:49.571373 16446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:49.572145 16446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:49.572216 16446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.572285 16446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:49.572352 16446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.580747 16446 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:37565
I20260621 07:56:49.580775 16573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:37565 every 8 connection(s)
I20260621 07:56:49.581238 16446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:49.583896 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16446
I20260621 07:56:49.584038 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:49.584400 12158 external_mini_cluster.cc:1428] Setting key fe42659193c4be012ef20326b574ae60
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:56:49.595184 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.582757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:60569 (local address 127.11.223.190:33995)
0621 07:56:49.582892 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.582896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.583620 (+ 724us) server_negotiation.cc:407] Connection header received
0621 07:56:49.585287 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.585290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.585365 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.585474 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:49.586902 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.587361 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.588495 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.588723 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.588832 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.591428 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:49.591431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:49.591434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:49.591465 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:49.593137 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.593624 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.593631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.593632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.593734 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.594097 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.594100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.594101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.594267 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:49.594370 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.594883 (+ 513us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.595006 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 07:56:49.596032 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.583094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33995 (local address 127.11.223.130:60569)
0621 07:56:49.583482 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.583518 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.585070 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.585640 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.585649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.586135 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:49.586762 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.586781 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.587488 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.587497 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.588381 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.588390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.588862 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.588866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.589031 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.589619 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:49.589634 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:49.591265 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:49.593247 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:49.593251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:49.593261 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:49.593518 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:49.593820 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:49.593822 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:49.593824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:49.593947 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:49.594378 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:49.594387 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:49.594599 (+ 212us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.594845 (+ 246us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.595136 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 07:56:49.597297 16574 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33995
I20260621 07:56:49.597527 16574 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:49.597931 16574 heartbeater.cc:511] Master 127.11.223.190:33995 requested a full tablet report, sending...
I20260621 07:56:49.598917 16251 ts_manager.cc:194] Registered new tserver with Master: cd8b2f48cc954c6caaff09d52abfa87b (127.11.223.130:37565)
I20260621 07:56:49.599469 16251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:60569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:49.611178 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33995
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:49.723002 16579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:49.723268 16579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:49.723366 16579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:49.727134 16579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:49.727209 16579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:49.727319 16579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:49.732546 16579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:49.733471 16579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:49.734368 16579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:49.743290 16584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.743330 16587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:49.743273 16585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:49.743467 16579 server_base.cc:1034] running on GCE node
I20260621 07:56:49.743927 16579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:49.744549 16579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:49.745725 16579 hybrid_clock.cc:648] HybridClock initialized: now 1782028609745712 us; error 25 us; skew 500 ppm
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:49.748301 16579 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:49.749044 16579 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:49.749152 16579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:49.749357 16579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:49.751233 16579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "6d004e74e56e430fa38ca011fbb37e9e"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "746970e02f9d32dbb92acc5c55ed2726"
server_key_iv: "91073498701819cc178ed97149dc5a63"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.751745 16579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "6d004e74e56e430fa38ca011fbb37e9e"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "746970e02f9d32dbb92acc5c55ed2726"
server_key_iv: "91073498701819cc178ed97149dc5a63"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.755103 16579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260621 07:56:49.757251 16593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.758261 16579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:49.758380 16579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "6d004e74e56e430fa38ca011fbb37e9e"
format_stamp: "Formatted at 2026-06-21 07:56:49 on dist-test-slave-0pn3"
server_key: "746970e02f9d32dbb92acc5c55ed2726"
server_key_iv: "91073498701819cc178ed97149dc5a63"
server_key_version: "encryptionkey@0"
I20260621 07:56:49.758476 16579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:49.773841 16579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:49.776492 16579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:49.776679 16579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:49.777199 16579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:49.777964 16579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:49.778031 16579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.778095 16579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:49.778147 16579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:49.786559 16579 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:36995
I20260621 07:56:49.786576 16706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:36995 every 8 connection(s)
I20260621 07:56:49.787076 16579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:49.790228 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16579
I20260621 07:56:49.790345 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:49.790616 12158 external_mini_cluster.cc:1428] Setting key 5e435aca05b718f19300e6767fc70d0c
Jun 21 07:56:49 dist-test-slave-0pn3 krb5kdc[16207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:49.799602 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.788707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:36717 (local address 127.11.223.190:33995)
0621 07:56:49.788851 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.788856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.789723 (+ 867us) server_negotiation.cc:407] Connection header received
0621 07:56:49.791103 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.791106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.791159 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.791249 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:49.792854 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.793371 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.794407 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.794614 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.794690 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.796800 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:49.796803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:49.796805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:49.796826 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:49.798020 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.798438 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.798440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.798442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.798510 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.798777 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.798779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.798780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.798940 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:49.799012 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.799404 (+ 392us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.799490 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 07:56:49.800388 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.789107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33995 (local address 127.11.223.131:36717)
0621 07:56:49.789511 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.789601 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.790848 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.791562 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.791568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.792065 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:49.792700 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.792714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.793471 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.793473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.794303 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.794311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.794729 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.794731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.794856 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.795335 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:49.795349 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:49.796663 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:49.798118 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:49.798122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:49.798132 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:49.798358 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:49.798585 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:49.798587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:49.798588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:49.798704 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:49.799022 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:49.799026 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:49.799222 (+ 196us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.799360 (+ 138us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.799580 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":102,"threads_started":1}
I20260621 07:56:49.801457 16707 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33995
I20260621 07:56:49.801677 16707 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:49.802098 16707 heartbeater.cc:511] Master 127.11.223.190:33995 requested a full tablet report, sending...
I20260621 07:56:49.803035 16251 ts_manager.cc:194] Registered new tserver with Master: 6d004e74e56e430fa38ca011fbb37e9e (127.11.223.131:36995)
I20260621 07:56:49.803639 16251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:36717
I20260621 07:56:49.804394 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:56:49.812809 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.806417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.11.223.190:33995)
0621 07:56:49.806530 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.806533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.806692 (+ 159us) server_negotiation.cc:407] Connection header received
0621 07:56:49.806782 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.806784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.806815 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.806870 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:49.807483 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.807933 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.808737 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.808930 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.809003 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.810165 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:49.810168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:49.810170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:49.810190 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:49.811522 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.811853 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.811855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.811856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.811922 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:49.812127 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:49.812129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:49.812130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:49.812301 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:49.812453 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.812635 (+ 182us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.812712 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:49.815330 16251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:49.818293 16251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:49.832161 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.826515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:37565 (local address 127.0.0.1:50182)
0621 07:56:49.827354 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.827366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.827449 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.828102 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.828105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.828118 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:49.828202 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.828209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.829441 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.829447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.830607 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.830617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.831614 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.831618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.831727 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.831767 (+ 40us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.831881 (+ 114us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.832007 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":94,"spinlock_wait_cycles":3200,"thread_start_us":101,"threads_started":1}
I20260621 07:56:49.832191 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.827128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50182 (local address 127.11.223.130:37565)
0621 07:56:49.827449 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.827461 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.827508 (+ 47us) server_negotiation.cc:407] Connection header received
0621 07:56:49.827586 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.827591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.827793 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.827943 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:49.828357 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.829162 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.830736 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.831504 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.831628 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.831757 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.831929 (+ 172us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.832027 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":118,"threads_started":1}
I20260621 07:56:49.832741 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.826870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56636 (local address 127.11.223.129:35989)
0621 07:56:49.827227 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.827234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.827369 (+ 135us) server_negotiation.cc:407] Connection header received
0621 07:56:49.827632 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.827636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.827786 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.827942 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:49.828412 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.829638 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.831146 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.831947 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.832052 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.832162 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.832496 (+ 334us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.832588 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":67,"threads_started":1}
I20260621 07:56:49.832652 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.826198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:35989 (local address 127.0.0.1:56636)
0621 07:56:49.827287 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.827300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.827446 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.827942 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.827947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.827968 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:49.828303 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.828309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.829829 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.829832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.830985 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.830995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.832225 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.832228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.832370 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.832391 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.832463 (+ 72us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.832525 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":1,"spinlock_wait_cycles":5376,"thread_start_us":75,"threads_started":1}
I20260621 07:56:49.833119 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.826130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36995 (local address 127.0.0.1:52216)
0621 07:56:49.826886 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.826945 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.827060 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.827962 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.827965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.827985 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:49.828156 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.828166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.829608 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.829612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.830933 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.830944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.832440 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.832445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.832537 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.832559 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.832630 (+ 71us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.832696 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":20,"thread_start_us":80,"threads_started":1}
I20260621 07:56:49.833407 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.826245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.11.223.131:36995)
0621 07:56:49.827547 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.827557 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.827573 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:56:49.827627 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.827631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.827793 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.828109 (+ 316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:49.828265 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.829467 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.831060 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.832089 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.832207 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.832364 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.833162 (+ 798us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.833263 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1218,"thread_start_us":157,"threads_started":1}
I20260621 07:56:49.835668 16375 tablet_service.cc:1467] Processing CreateTablet for tablet fd70df995cb748969d49dbaae22aa5e6 (DEFAULT_TABLE table=test-table [id=fae533a95ef74c10a7898bf46bc58a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:49.835855 16641 tablet_service.cc:1467] Processing CreateTablet for tablet fd70df995cb748969d49dbaae22aa5e6 (DEFAULT_TABLE table=test-table [id=fae533a95ef74c10a7898bf46bc58a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:49.836741 16641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd70df995cb748969d49dbaae22aa5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:49.836737 16375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd70df995cb748969d49dbaae22aa5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:49.837855 16508 tablet_service.cc:1467] Processing CreateTablet for tablet fd70df995cb748969d49dbaae22aa5e6 (DEFAULT_TABLE table=test-table [id=fae533a95ef74c10a7898bf46bc58a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:49.838799 16508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd70df995cb748969d49dbaae22aa5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:49.842793 16724 tablet_bootstrap.cc:492] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Bootstrap starting.
I20260621 07:56:49.844645 16725 tablet_bootstrap.cc:492] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: Bootstrap starting.
I20260621 07:56:49.844933 16724 tablet_bootstrap.cc:654] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:49.845206 16726 tablet_bootstrap.cc:492] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Bootstrap starting.
I20260621 07:56:49.846226 16724 log.cc:824] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:49.846488 16725 tablet_bootstrap.cc:654] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:49.847642 16725 log.cc:824] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:49.847728 16726 tablet_bootstrap.cc:654] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:49.848194 16724 tablet_bootstrap.cc:492] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: No bootstrap required, opened a new log
I20260621 07:56:49.848425 16724 ts_tablet_manager.cc:1397] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 07:56:49.848579 16726 log.cc:824] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:49.849660 16725 tablet_bootstrap.cc:492] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: No bootstrap required, opened a new log
I20260621 07:56:49.849843 16725 ts_tablet_manager.cc:1397] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:56:49.850242 16726 tablet_bootstrap.cc:492] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: No bootstrap required, opened a new log
I20260621 07:56:49.850430 16726 ts_tablet_manager.cc:1397] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:56:49.852185 16724 raft_consensus.cc:348] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.852464 16724 raft_consensus.cc:374] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:49.852543 16724 raft_consensus.cc:729] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d004e74e56e430fa38ca011fbb37e9e, State: Initialized, Role: FOLLOWER
I20260621 07:56:49.852967 16724 consensus_queue.cc:260] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.853161 16726 raft_consensus.cc:348] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.853461 16726 raft_consensus.cc:374] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:49.853402 16725 raft_consensus.cc:348] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.853572 16726 raft_consensus.cc:729] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8b2f48cc954c6caaff09d52abfa87b, State: Initialized, Role: FOLLOWER
I20260621 07:56:49.853593 16707 heartbeater.cc:503] Master 127.11.223.190:33995 was elected leader, sending a full tablet report...
I20260621 07:56:49.853711 16725 raft_consensus.cc:374] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:49.853803 16725 raft_consensus.cc:729] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209fec5a2b174b1aadd400988d784a1d, State: Initialized, Role: FOLLOWER
I20260621 07:56:49.853945 16724 ts_tablet_manager.cc:1428] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 07:56:49.853977 16726 consensus_queue.cc:260] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.854250 16725 consensus_queue.cc:260] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.854636 16574 heartbeater.cc:503] Master 127.11.223.190:33995 was elected leader, sending a full tablet report...
I20260621 07:56:49.854893 16726 ts_tablet_manager.cc:1428] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260621 07:56:49.855152 16725 ts_tablet_manager.cc:1428] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 07:56:49.855472 16441 heartbeater.cc:503] Master 127.11.223.190:33995 was elected leader, sending a full tablet report...
W20260621 07:56:49.857462 16442 tablet.cc:2369] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:49.910468 16732 raft_consensus.cc:482] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:49.910679 16732 raft_consensus.cc:504] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.911726 16732 leader_election.cc:288] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d004e74e56e430fa38ca011fbb37e9e (127.11.223.131:36995), cd8b2f48cc954c6caaff09d52abfa87b (127.11.223.130:37565)
I20260621 07:56:49.915437 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.912168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:35495 (local address 127.11.223.131:36995)
0621 07:56:49.912290 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.912295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.912461 (+ 166us) server_negotiation.cc:407] Connection header received
0621 07:56:49.912607 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.912610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.912659 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.912733 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:49.913014 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.913494 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.914544 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.914922 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.915044 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.915096 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.915256 (+ 160us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.915316 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 07:56:49.915449 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.912038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36995 (local address 127.11.223.129:35495)
0621 07:56:49.912372 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.912384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.912504 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.912753 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.912755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.912771 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:49.912890 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.912906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.913598 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.913604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.914426 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.914434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.915037 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.915040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.915133 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.915154 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.915243 (+ 89us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.915308 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
I20260621 07:56:49.915540 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.912172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:39863 (local address 127.11.223.130:37565)
0621 07:56:49.912375 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.912380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.912547 (+ 167us) server_negotiation.cc:407] Connection header received
0621 07:56:49.912704 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.912717 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.912756 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.912836 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:49.913098 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.913574 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.914606 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.914994 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.915116 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.915167 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.915350 (+ 183us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.915419 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 07:56:49.915593 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.912038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:37565 (local address 127.11.223.129:39863)
0621 07:56:49.912464 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.912475 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:49.912583 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:49.912847 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:49.912849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:49.912864 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:49.912993 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.912998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.913687 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.913691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.914477 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:49.914485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.915111 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.915114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:49.915216 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.915235 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 07:56:49.915325 (+ 90us) client_negotiation.cc:240] Negotiation successful
0621 07:56:49.915393 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":12,"thread_start_us":73,"threads_started":1}
I20260621 07:56:49.916222 16661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd70df995cb748969d49dbaae22aa5e6" candidate_uuid: "209fec5a2b174b1aadd400988d784a1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d004e74e56e430fa38ca011fbb37e9e" is_pre_election: true
I20260621 07:56:49.916307 16528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd70df995cb748969d49dbaae22aa5e6" candidate_uuid: "209fec5a2b174b1aadd400988d784a1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" is_pre_election: true
I20260621 07:56:49.916617 16661 raft_consensus.cc:2454] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 209fec5a2b174b1aadd400988d784a1d in term 0.
I20260621 07:56:49.916639 16528 raft_consensus.cc:2454] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 209fec5a2b174b1aadd400988d784a1d in term 0.
I20260621 07:56:49.917106 16329 leader_election.cc:302] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209fec5a2b174b1aadd400988d784a1d, 6d004e74e56e430fa38ca011fbb37e9e; no voters:
I20260621 07:56:49.917404 16732 raft_consensus.cc:2781] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:49.917511 16732 raft_consensus.cc:482] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:49.917586 16732 raft_consensus.cc:3037] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:49.918555 16732 raft_consensus.cc:504] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.918893 16732 leader_election.cc:288] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [CANDIDATE]: Term 1 election: Requested vote from peers 6d004e74e56e430fa38ca011fbb37e9e (127.11.223.131:36995), cd8b2f48cc954c6caaff09d52abfa87b (127.11.223.130:37565)
I20260621 07:56:49.919222 16661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd70df995cb748969d49dbaae22aa5e6" candidate_uuid: "209fec5a2b174b1aadd400988d784a1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d004e74e56e430fa38ca011fbb37e9e"
I20260621 07:56:49.919258 16528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd70df995cb748969d49dbaae22aa5e6" candidate_uuid: "209fec5a2b174b1aadd400988d784a1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8b2f48cc954c6caaff09d52abfa87b"
I20260621 07:56:49.919317 16661 raft_consensus.cc:3037] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:49.919390 16528 raft_consensus.cc:3037] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:49.920382 16661 raft_consensus.cc:2454] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 209fec5a2b174b1aadd400988d784a1d in term 1.
I20260621 07:56:49.920527 16528 raft_consensus.cc:2454] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 209fec5a2b174b1aadd400988d784a1d in term 1.
I20260621 07:56:49.920662 16329 leader_election.cc:302] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 209fec5a2b174b1aadd400988d784a1d, 6d004e74e56e430fa38ca011fbb37e9e; no voters:
I20260621 07:56:49.920816 16732 raft_consensus.cc:2781] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:49.921000 16732 raft_consensus.cc:686] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [term 1 LEADER]: Becoming Leader. State: Replica: 209fec5a2b174b1aadd400988d784a1d, State: Running, Role: LEADER
I20260621 07:56:49.921516 16732 consensus_queue.cc:237] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } }
I20260621 07:56:49.924503 16248 catalog_manager.cc:5495] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 209fec5a2b174b1aadd400988d784a1d (127.11.223.129). New cstate: current_term: 1 leader_uuid: "209fec5a2b174b1aadd400988d784a1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "209fec5a2b174b1aadd400988d784a1d" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 } health_report { overall_health: UNKNOWN } } }
I20260621 07:56:49.953939 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.950484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.11.223.129:35989)
0621 07:56:49.950609 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:49.950613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:49.950627 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:56:49.950759 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:49.950762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:49.950812 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:49.950930 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:49.951168 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.951670 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.952565 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:49.952777 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:49.952883 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:49.953178 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:49.953554 (+ 376us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:49.953680 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:49.953741 (+ 61us) server_negotiation.cc:299] Negotiation successful
0621 07:56:49.953794 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260621 07:56:49.959539 16661 raft_consensus.cc:1261] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 1 FOLLOWER]: Refusing update from remote peer 209fec5a2b174b1aadd400988d784a1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:49.959595 16528 raft_consensus.cc:1261] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 1 FOLLOWER]: Refusing update from remote peer 209fec5a2b174b1aadd400988d784a1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:49.960229 16735 consensus_queue.cc:1035] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d004e74e56e430fa38ca011fbb37e9e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:49.960456 16732 consensus_queue.cc:1035] T fd70df995cb748969d49dbaae22aa5e6 P 209fec5a2b174b1aadd400988d784a1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd8b2f48cc954c6caaff09d52abfa87b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:49.974653 16251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:49.974740 16742 mvcc.cc:205] Tried to move back new op lower bound from 7299189186387386368 to 7299189186241175552. Current Snapshot: MvccSnapshot[applied={T|T < 7299189186387386368}]
I20260621 07:56:49.974818 16251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:49.977037 16251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02ec7f8c1a1445d2be272d358ae4ac68: Sending DeleteTablet for 3 replicas of tablet fd70df995cb748969d49dbaae22aa5e6
I20260621 07:56:49.977723 16641 tablet_service.cc:1514] Processing DeleteTablet for tablet fd70df995cb748969d49dbaae22aa5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:49 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:52216
I20260621 07:56:49.977844 16508 tablet_service.cc:1514] Processing DeleteTablet for tablet fd70df995cb748969d49dbaae22aa5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:49 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50182
I20260621 07:56:49.978279 16748 tablet_replica.cc:330] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: stopping tablet replica
I20260621 07:56:49.978384 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16313
I20260621 07:56:49.978502 16747 tablet_replica.cc:330] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: stopping tablet replica
I20260621 07:56:49.979475 16747 raft_consensus.cc:2229] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:49.979761 16747 raft_consensus.cc:2258] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:49.980852 16748 raft_consensus.cc:2229] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:49.981107 16748 raft_consensus.cc:2258] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:49.981186 16747 ts_tablet_manager.cc:1905] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:49.982187 16748 ts_tablet_manager.cc:1905] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:49.984241 16748 ts_tablet_manager.cc:1918] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:49.984367 16748 log.cc:1196] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/fd70df995cb748969d49dbaae22aa5e6
I20260621 07:56:49.984612 16747 ts_tablet_manager.cc:1918] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:49.984699 16748 ts_tablet_manager.cc:1939] T fd70df995cb748969d49dbaae22aa5e6 P cd8b2f48cc954c6caaff09d52abfa87b: Deleting consensus metadata
I20260621 07:56:49.984752 16747 log.cc:1196] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/fd70df995cb748969d49dbaae22aa5e6
I20260621 07:56:49.985116 16747 ts_tablet_manager.cc:1939] T fd70df995cb748969d49dbaae22aa5e6 P 6d004e74e56e430fa38ca011fbb37e9e: Deleting consensus metadata
W20260621 07:56:49.985783 16235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:35989 (error 108)
I20260621 07:56:49.985864 16237 catalog_manager.cc:4841] TS cd8b2f48cc954c6caaff09d52abfa87b (127.11.223.130:37565): tablet fd70df995cb748969d49dbaae22aa5e6 (table test-table [id=fae533a95ef74c10a7898bf46bc58a0c]) successfully deleted
I20260621 07:56:49.986169 16236 catalog_manager.cc:4841] TS 6d004e74e56e430fa38ca011fbb37e9e (127.11.223.131:36995): tablet fd70df995cb748969d49dbaae22aa5e6 (table test-table [id=fae533a95ef74c10a7898bf46bc58a0c]) successfully deleted
I20260621 07:56:49.986783 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16446
I20260621 07:56:49.987066 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:49.986778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:35989 (local address 127.0.0.1:56648)
0621 07:56:49.986901 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:49.986980 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260621 07:56:49.987356 16235 catalog_manager.cc:4584] TS 209fec5a2b174b1aadd400988d784a1d (127.11.223.129:35989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd70df995cb748969d49dbaae22aa5e6: Network error: Client connection negotiation failed: client connection to 127.11.223.129:35989: connect: Connection refused (error 111)
I20260621 07:56:49.992064 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16579
I20260621 07:56:49.997296 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16220
2026-06-21T07:56:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2195 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-21T07:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:50Z Disabled control of system clock
I20260621 07:56:50.025985 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:42789
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:42789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260621 07:56:50.129510 16753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:50.129742 16753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:50.129797 16753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:50.133764 16753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:50.133812 16753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:50.133828 16753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:50.133844 16753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:50.133859 16753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:50.138263 16753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:42789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:42789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:50.139113 16753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:50.140412 16753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:50.148487 16759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:50.148494 16758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:50.148554 16761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:50.148527 16753 server_base.cc:1034] running on GCE node
I20260621 07:56:50.149273 16753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:50.150332 16753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:50.151494 16753 hybrid_clock.cc:648] HybridClock initialized: now 1782028610151476 us; error 32 us; skew 500 ppm
I20260621 07:56:50.153304 16753 webserver.cc:460] Webserver started at http://127.11.223.190:37927/ using document root <none> and password file <none>
I20260621 07:56:50.153891 16753 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:50.153951 16753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:50.154152 16753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:50.156001 16753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "2e544535a7f84c0d8cd904f81819cc61"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "836eb57eb4b1a73b59749ea1ed7ca30a"
server_key_iv: "7214bf846b80cd50e837e6584e92374c"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.156553 16753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "2e544535a7f84c0d8cd904f81819cc61"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "836eb57eb4b1a73b59749ea1ed7ca30a"
server_key_iv: "7214bf846b80cd50e837e6584e92374c"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.159889 16753 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260621 07:56:50.162094 16767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.163203 16753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:50.163355 16753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "2e544535a7f84c0d8cd904f81819cc61"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "836eb57eb4b1a73b59749ea1ed7ca30a"
server_key_iv: "7214bf846b80cd50e837e6584e92374c"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.163460 16753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:50.167610 16753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:50.168140 16753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:50.168357 16753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:50.174458 16753 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:42789
I20260621 07:56:50.174472 16819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:42789 every 8 connection(s)
I20260621 07:56:50.175292 16753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:50.178242 16820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:50.181241 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16753
I20260621 07:56:50.181368 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:50.181613 12158 external_mini_cluster.cc:1428] Setting key a9449f549e9b8d11735eb48bc7568920
I20260621 07:56:50.183686 16820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61: Bootstrap starting.
I20260621 07:56:50.185823 16820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:50.186631 16820 log.cc:824] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:50.186765 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.182906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59356 (local address 127.11.223.190:42789)
0621 07:56:50.183487 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.183501 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.183531 (+ 30us) server_negotiation.cc:407] Connection header received
0621 07:56:50.184167 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.184182 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.184558 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.184869 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.185610 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.185615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.185625 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.185634 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.185683 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.185697 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.185815 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.185943 (+ 128us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.186161 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":123,"threads_started":1}
I20260621 07:56:50.188580 16820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61: No bootstrap required, opened a new log
I20260621 07:56:50.191789 16820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e544535a7f84c0d8cd904f81819cc61" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42789 } }
I20260621 07:56:50.192025 16820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:50.192075 16820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e544535a7f84c0d8cd904f81819cc61, State: Initialized, Role: FOLLOWER
I20260621 07:56:50.192443 16820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e544535a7f84c0d8cd904f81819cc61" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42789 } }
I20260621 07:56:50.192554 16820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:50.192596 16820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:50.192677 16820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:50.193622 16820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e544535a7f84c0d8cd904f81819cc61" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42789 } }
I20260621 07:56:50.193913 16820 leader_election.cc:302] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e544535a7f84c0d8cd904f81819cc61; no voters:
I20260621 07:56:50.194141 16820 leader_election.cc:288] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:50.194240 16825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:50.194429 16825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [term 1 LEADER]: Becoming Leader. State: Replica: 2e544535a7f84c0d8cd904f81819cc61, State: Running, Role: LEADER
I20260621 07:56:50.194715 16825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e544535a7f84c0d8cd904f81819cc61" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42789 } }
I20260621 07:56:50.195219 16820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:50.196167 16827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e544535a7f84c0d8cd904f81819cc61. Latest consensus state: current_term: 1 leader_uuid: "2e544535a7f84c0d8cd904f81819cc61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e544535a7f84c0d8cd904f81819cc61" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42789 } } }
I20260621 07:56:50.196285 16827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:50.196624 16826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e544535a7f84c0d8cd904f81819cc61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e544535a7f84c0d8cd904f81819cc61" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42789 } } }
I20260621 07:56:50.196732 16826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:50.196748 16834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:50.199259 16834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:50.204331 16834 catalog_manager.cc:1347] Generated new cluster ID: 7115f434565a4271869a836c4cf4276b
I20260621 07:56:50.204392 16834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:50.232604 16834 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:50.232919 16834 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:50.248859 16834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61: Generated new TSK 0
I20260621 07:56:50.249598 16834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:56:50.256268 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42789
--builtin_ntp_servers=127.11.223.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260621 07:56:50.359740 16844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:50.360014 16844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:50.360086 16844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:50.364034 16844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:50.364114 16844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:50.364202 16844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:50.368793 16844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:50.369818 16844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:50.371110 16844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:50.380016 16849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:50.380009 16852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:50.380110 16844 server_base.cc:1034] running on GCE node
W20260621 07:56:50.380010 16850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:50.380611 16844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:50.381182 16844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:50.382335 16844 hybrid_clock.cc:648] HybridClock initialized: now 1782028610382323 us; error 24 us; skew 500 ppm
I20260621 07:56:50.384145 16844 webserver.cc:460] Webserver started at http://127.11.223.129:44033/ using document root <none> and password file <none>
I20260621 07:56:50.384733 16844 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:50.384792 16844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:50.384986 16844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:50.386881 16844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "2ceb5a700eed4f66a6010389bdc298af"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "d4d9813b80086a1a29f137e92bd6b343"
server_key_iv: "c1df9471efca2d213c8bf519f5ddebe0"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.387406 16844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "2ceb5a700eed4f66a6010389bdc298af"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "d4d9813b80086a1a29f137e92bd6b343"
server_key_iv: "c1df9471efca2d213c8bf519f5ddebe0"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.390808 16844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260621 07:56:50.393210 16858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.394297 16844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:50.394416 16844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "2ceb5a700eed4f66a6010389bdc298af"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "d4d9813b80086a1a29f137e92bd6b343"
server_key_iv: "c1df9471efca2d213c8bf519f5ddebe0"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.394512 16844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:50.398927 16844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:50.399490 16844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:50.399662 16844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:50.400167 16844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:50.401057 16844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:50.401104 16844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.401166 16844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:50.401218 16844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.409117 16844 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:45591
I20260621 07:56:50.409224 16971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:45591 every 8 connection(s)
I20260621 07:56:50.410009 16844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:50.411580 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16844
I20260621 07:56:50.411695 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:50.411998 12158 external_mini_cluster.cc:1428] Setting key fef3ab11aa22403003db1dc301fc9969
I20260621 07:56:50.413908 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42789
--builtin_ntp_servers=127.11.223.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260621 07:56:50.416042 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.411714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:40091 (local address 127.11.223.190:42789)
0621 07:56:50.411849 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.411853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.412955 (+ 1102us) server_negotiation.cc:407] Connection header received
0621 07:56:50.414271 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.414274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.414316 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.414406 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.415599 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.415602 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.415605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.415615 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.415637 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.415644 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.415715 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.415907 (+ 192us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.415944 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:56:50.416985 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.412111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:42789 (local address 127.11.223.129:40091)
0621 07:56:50.412793 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.412841 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.414040 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.414573 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.414579 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.414671 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.415295 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.415298 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.415350 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.415442 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.415735 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.415745 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.415753 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.415943 (+ 190us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.416204 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":146,"threads_started":1}
I20260621 07:56:50.418135 16972 heartbeater.cc:347] Connected to a master server at 127.11.223.190:42789
I20260621 07:56:50.418361 16972 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:50.418736 16972 heartbeater.cc:511] Master 127.11.223.190:42789 requested a full tablet report, sending...
I20260621 07:56:50.420241 16784 ts_manager.cc:194] Registered new tserver with Master: 2ceb5a700eed4f66a6010389bdc298af (127.11.223.129:45591)
W20260621 07:56:50.522149 16975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:50.522418 16975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:50.522490 16975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:50.526216 16975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:50.526291 16975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:50.526386 16975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:50.530977 16975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:50.532003 16975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:50.533432 16975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:50.542649 16981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:50.542681 16975 server_base.cc:1034] running on GCE node
W20260621 07:56:50.542649 16983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:50.542683 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:50.543313 16975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:50.543876 16975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:50.545081 16975 hybrid_clock.cc:648] HybridClock initialized: now 1782028610545059 us; error 41 us; skew 500 ppm
I20260621 07:56:50.546759 16975 webserver.cc:460] Webserver started at http://127.11.223.130:33943/ using document root <none> and password file <none>
I20260621 07:56:50.547307 16975 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:50.547390 16975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:50.547591 16975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:50.549551 16975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "dbeedbfef8624b18b2d5340146daa883"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "f69cc67c967088ca589d791be389b1a3"
server_key_iv: "69f23c48b37cb4531baf09d01fb3e007"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.550104 16975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "dbeedbfef8624b18b2d5340146daa883"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "f69cc67c967088ca589d791be389b1a3"
server_key_iv: "69f23c48b37cb4531baf09d01fb3e007"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.553469 16975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260621 07:56:50.555722 16989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.556927 16975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:56:50.557053 16975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "dbeedbfef8624b18b2d5340146daa883"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "f69cc67c967088ca589d791be389b1a3"
server_key_iv: "69f23c48b37cb4531baf09d01fb3e007"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.557158 16975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:50.561375 16975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:50.561959 16975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:50.562134 16975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:50.562633 16975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:50.563472 16975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:50.563540 16975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.563609 16975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:50.563660 16975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.572227 16975 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:39979
I20260621 07:56:50.572253 17102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:39979 every 8 connection(s)
I20260621 07:56:50.573098 16975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:50.578821 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.574810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:52629 (local address 127.11.223.190:42789)
0621 07:56:50.574952 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.574956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.575850 (+ 894us) server_negotiation.cc:407] Connection header received
0621 07:56:50.577137 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.577140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.577187 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.577263 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.578338 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.578341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.578344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.578356 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.578378 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.578401 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.578493 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.578644 (+ 151us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.578689 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:56:50.579579 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.575111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:42789 (local address 127.11.223.130:52629)
0621 07:56:50.575701 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.575736 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.576932 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.577427 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.577433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.577515 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.578089 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.578091 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.578128 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.578190 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.578489 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.578497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.578505 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.578673 (+ 168us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.578899 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260621 07:56:50.579907 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 16975
I20260621 07:56:50.580054 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:50.580436 12158 external_mini_cluster.cc:1428] Setting key dcb6ec56bc5aa2e072b75331c9a39b89
I20260621 07:56:50.580746 17103 heartbeater.cc:347] Connected to a master server at 127.11.223.190:42789
I20260621 07:56:50.580988 17103 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:50.581380 17103 heartbeater.cc:511] Master 127.11.223.190:42789 requested a full tablet report, sending...
I20260621 07:56:50.582383 16784 ts_manager.cc:194] Registered new tserver with Master: dbeedbfef8624b18b2d5340146daa883 (127.11.223.130:39979)
I20260621 07:56:50.582376 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42789
--builtin_ntp_servers=127.11.223.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260621 07:56:50.690065 17106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:50.690303 17106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:50.690343 17106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:50.694226 17106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:50.694274 17106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:50.694327 17106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:50.699342 17106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:50.700435 17106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:50.701670 17106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:50.710738 17114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:50.710759 17111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:50.710738 17112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:50.711076 17106 server_base.cc:1034] running on GCE node
I20260621 07:56:50.711483 17106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:50.712081 17106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:50.713264 17106 hybrid_clock.cc:648] HybridClock initialized: now 1782028610713243 us; error 35 us; skew 500 ppm
I20260621 07:56:50.714939 17106 webserver.cc:460] Webserver started at http://127.11.223.131:41089/ using document root <none> and password file <none>
I20260621 07:56:50.715508 17106 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:50.715597 17106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:50.715806 17106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:50.717803 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "6f451c105fdc4f048e1131b7cce29d52"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "4d0a3adabae4ef45f1717296f43f2505"
server_key_iv: "95304f4f5fcf303bd84e35b932cd51c9"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.718333 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "6f451c105fdc4f048e1131b7cce29d52"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "4d0a3adabae4ef45f1717296f43f2505"
server_key_iv: "95304f4f5fcf303bd84e35b932cd51c9"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.721782 17106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:50.724048 17120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.725261 17106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:50.725400 17106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "6f451c105fdc4f048e1131b7cce29d52"
format_stamp: "Formatted at 2026-06-21 07:56:50 on dist-test-slave-0pn3"
server_key: "4d0a3adabae4ef45f1717296f43f2505"
server_key_iv: "95304f4f5fcf303bd84e35b932cd51c9"
server_key_version: "encryptionkey@0"
I20260621 07:56:50.725512 17106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:50.729774 17106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:50.730342 17106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:50.730535 17106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:50.731033 17106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:50.731917 17106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:50.731986 17106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.732052 17106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:50.732100 17106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:50.740676 17106 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:36025
I20260621 07:56:50.740702 17233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:36025 every 8 connection(s)
I20260621 07:56:50.741501 17106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:50.747208 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.743137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:34571 (local address 127.11.223.190:42789)
0621 07:56:50.743270 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.743274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.744227 (+ 953us) server_negotiation.cc:407] Connection header received
0621 07:56:50.745485 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.745489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.745533 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.745642 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.746724 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.746728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.746730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.746743 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.746772 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.746782 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.746875 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.747030 (+ 155us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.747076 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:56:50.747969 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.743465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:42789 (local address 127.11.223.131:34571)
0621 07:56:50.744076 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.744114 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.745275 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.745800 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.745807 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.745895 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.746469 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.746471 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.746509 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.746570 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.746869 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.746885 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.746893 (+ 8us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.747059 (+ 166us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.747280 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":108,"threads_started":1}
I20260621 07:56:50.748318 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17106
I20260621 07:56:50.748452 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:50.748773 12158 external_mini_cluster.cc:1428] Setting key 672010f090cec56fdb5b58bcde150f2f
I20260621 07:56:50.749130 17234 heartbeater.cc:347] Connected to a master server at 127.11.223.190:42789
I20260621 07:56:50.749357 17234 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:50.749811 17234 heartbeater.cc:511] Master 127.11.223.190:42789 requested a full tablet report, sending...
I20260621 07:56:50.750828 16784 ts_manager.cc:194] Registered new tserver with Master: 6f451c105fdc4f048e1131b7cce29d52 (127.11.223.131:36025)
I20260621 07:56:50.762336 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:56:50.765574 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.764074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.11.223.190:42789)
0621 07:56:50.764208 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.764214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.764362 (+ 148us) server_negotiation.cc:407] Connection header received
0621 07:56:50.764497 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.764499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.764530 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.764600 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.765237 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.765239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.765241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.765248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.765261 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.765267 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.765321 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.765380 (+ 59us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.765416 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:56:50.768241 16784 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:56:50.771179 16784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:50.782445 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.779244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36025 (local address 127.0.0.1:48876)
0621 07:56:50.779973 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.779988 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.780057 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.780889 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.780896 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.780966 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.781538 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.781543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.781566 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.781635 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.782067 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.782077 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.782083 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.782197 (+ 114us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.782265 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":83,"threads_started":1}
I20260621 07:56:50.782470 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.779147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.11.223.130:39979)
0621 07:56:50.779472 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.779483 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.779832 (+ 349us) server_negotiation.cc:407] Connection header received
0621 07:56:50.780054 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.780061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.780251 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.780395 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.781974 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.781981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.781992 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.782005 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.782051 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.782089 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.782197 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.782245 (+ 48us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.782314 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":127,"threads_started":1}
I20260621 07:56:50.782605 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.779674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:45591 (local address 127.0.0.1:38388)
0621 07:56:50.780636 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.780650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.780729 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.781093 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.781096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.781112 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.782005 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.782009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.782020 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.782043 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.782382 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.782384 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.782398 (+ 14us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.782468 (+ 70us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.782501 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":87,"threads_started":1}
I20260621 07:56:50.782847 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.778971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:39979 (local address 127.0.0.1:35436)
0621 07:56:50.779708 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.779752 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.779923 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.781103 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.781107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.781125 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.781769 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.781771 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.781780 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.781802 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.782166 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.782169 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.782172 (+ 3us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.782236 (+ 64us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.782273 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":87,"threads_started":1}
I20260621 07:56:50.783004 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.779838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.11.223.129:45591)
0621 07:56:50.780298 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.780332 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.780730 (+ 398us) server_negotiation.cc:407] Connection header received
0621 07:56:50.780813 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.780816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.780963 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.781100 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.782155 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.782161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.782171 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.782183 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.782224 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.782255 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.782344 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.782759 (+ 415us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.782842 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":99,"threads_started":1}
I20260621 07:56:50.783406 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.779732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.11.223.131:36025)
0621 07:56:50.780160 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.780170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.780191 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:56:50.780235 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.780239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.780433 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.781228 (+ 795us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.781833 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.781842 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.781855 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.781866 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.781909 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.781937 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.782069 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.782355 (+ 286us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.782410 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":150,"threads_started":1}
I20260621 07:56:50.785590 16906 tablet_service.cc:1467] Processing CreateTablet for tablet 5898386c014342e1b9df7e7fa9b5930a (DEFAULT_TABLE table=test-table [id=72ceba5cb1c54b48a82d4d00cc0f00be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:50.785826 17168 tablet_service.cc:1467] Processing CreateTablet for tablet 5898386c014342e1b9df7e7fa9b5930a (DEFAULT_TABLE table=test-table [id=72ceba5cb1c54b48a82d4d00cc0f00be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:50.785703 17037 tablet_service.cc:1467] Processing CreateTablet for tablet 5898386c014342e1b9df7e7fa9b5930a (DEFAULT_TABLE table=test-table [id=72ceba5cb1c54b48a82d4d00cc0f00be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:50.786624 16906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898386c014342e1b9df7e7fa9b5930a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:50.786669 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898386c014342e1b9df7e7fa9b5930a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:50.786669 17168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898386c014342e1b9df7e7fa9b5930a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:50.792497 17251 tablet_bootstrap.cc:492] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af: Bootstrap starting.
I20260621 07:56:50.794121 17252 tablet_bootstrap.cc:492] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Bootstrap starting.
I20260621 07:56:50.794706 17251 tablet_bootstrap.cc:654] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:50.795575 17251 log.cc:824] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:50.795616 17253 tablet_bootstrap.cc:492] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Bootstrap starting.
I20260621 07:56:50.796829 17252 tablet_bootstrap.cc:654] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:50.797581 17253 tablet_bootstrap.cc:654] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:50.797578 17251 tablet_bootstrap.cc:492] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af: No bootstrap required, opened a new log
I20260621 07:56:50.797570 17252 log.cc:824] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:50.797783 17251 ts_tablet_manager.cc:1397] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260621 07:56:50.798331 17253 log.cc:824] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:50.799487 17252 tablet_bootstrap.cc:492] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: No bootstrap required, opened a new log
I20260621 07:56:50.799667 17252 ts_tablet_manager.cc:1397] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260621 07:56:50.800062 17253 tablet_bootstrap.cc:492] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: No bootstrap required, opened a new log
I20260621 07:56:50.800243 17253 ts_tablet_manager.cc:1397] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 07:56:50.802233 17251 raft_consensus.cc:348] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.802551 17251 raft_consensus.cc:374] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:50.802619 17251 raft_consensus.cc:729] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ceb5a700eed4f66a6010389bdc298af, State: Initialized, Role: FOLLOWER
I20260621 07:56:50.803109 17251 consensus_queue.cc:260] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.803527 17253 raft_consensus.cc:348] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.803763 17253 raft_consensus.cc:374] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:50.803826 17253 raft_consensus.cc:729] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f451c105fdc4f048e1131b7cce29d52, State: Initialized, Role: FOLLOWER
I20260621 07:56:50.804054 17251 ts_tablet_manager.cc:1428] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260621 07:56:50.803928 17252 raft_consensus.cc:348] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.804111 16972 heartbeater.cc:503] Master 127.11.223.190:42789 was elected leader, sending a full tablet report...
I20260621 07:56:50.804142 17252 raft_consensus.cc:374] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:50.804184 17252 raft_consensus.cc:729] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbeedbfef8624b18b2d5340146daa883, State: Initialized, Role: FOLLOWER
I20260621 07:56:50.804251 17253 consensus_queue.cc:260] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.804597 17252 consensus_queue.cc:260] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.804901 17234 heartbeater.cc:503] Master 127.11.223.190:42789 was elected leader, sending a full tablet report...
I20260621 07:56:50.805456 17252 ts_tablet_manager.cc:1428] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260621 07:56:50.805546 17103 heartbeater.cc:503] Master 127.11.223.190:42789 was elected leader, sending a full tablet report...
I20260621 07:56:50.806452 17253 ts_tablet_manager.cc:1428] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260621 07:56:50.824121 17104 tablet.cc:2369] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:50.826321 17257 raft_consensus.cc:482] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:50.826509 17257 raft_consensus.cc:504] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.827610 17257 leader_election.cc:288] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbeedbfef8624b18b2d5340146daa883 (127.11.223.130:39979), 6f451c105fdc4f048e1131b7cce29d52 (127.11.223.131:36025)
I20260621 07:56:50.829563 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.827972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:38875 (local address 127.11.223.130:39979)
0621 07:56:50.828110 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.828115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.828129 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:56:50.828174 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.828177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.828212 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.828358 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.829081 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.829085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.829087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.829119 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.829160 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.829173 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.829277 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.829421 (+ 144us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.829460 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:56:50.829563 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.827784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:39979 (local address 127.11.223.129:38875)
0621 07:56:50.827928 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.827945 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.828029 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.828390 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.828396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.828415 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.828944 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.828946 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.828958 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.828987 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.829301 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.829304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.829307 (+ 3us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.829406 (+ 99us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.829445 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260621 07:56:50.830034 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.828003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:33179 (local address 127.11.223.131:36025)
0621 07:56:50.828146 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.828152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.828427 (+ 275us) server_negotiation.cc:407] Connection header received
0621 07:56:50.828593 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.828596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.828635 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.828704 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.829647 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.829652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.829655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.829668 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.829688 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.829698 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.829792 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.829903 (+ 111us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.829943 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:56:50.830152 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898386c014342e1b9df7e7fa9b5930a" candidate_uuid: "2ceb5a700eed4f66a6010389bdc298af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeedbfef8624b18b2d5340146daa883" is_pre_election: true
I20260621 07:56:50.830349 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.827873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36025 (local address 127.11.223.129:33179)
0621 07:56:50.828347 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.828361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:50.828482 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:50.828938 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:50.828942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:50.828958 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:50.829504 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:56:50.829506 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:50.829514 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:56:50.829535 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:50.829786 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:50.829789 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:50.829792 (+ 3us) client_negotiation.cc:769] Sending connection context
0621 07:56:50.830184 (+ 392us) client_negotiation.cc:240] Negotiation successful
0621 07:56:50.830223 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":121,"threads_started":1}
I20260621 07:56:50.830444 17057 raft_consensus.cc:2454] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ceb5a700eed4f66a6010389bdc298af in term 0.
I20260621 07:56:50.830873 16861 leader_election.cc:302] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ceb5a700eed4f66a6010389bdc298af, dbeedbfef8624b18b2d5340146daa883; no voters:
I20260621 07:56:50.830868 17188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898386c014342e1b9df7e7fa9b5930a" candidate_uuid: "2ceb5a700eed4f66a6010389bdc298af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f451c105fdc4f048e1131b7cce29d52" is_pre_election: true
I20260621 07:56:50.831131 17188 raft_consensus.cc:2454] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ceb5a700eed4f66a6010389bdc298af in term 0.
I20260621 07:56:50.831141 17257 raft_consensus.cc:2781] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:50.831234 17257 raft_consensus.cc:482] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:50.831306 17257 raft_consensus.cc:3037] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:50.832386 17257 raft_consensus.cc:504] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.832757 17257 leader_election.cc:288] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [CANDIDATE]: Term 1 election: Requested vote from peers dbeedbfef8624b18b2d5340146daa883 (127.11.223.130:39979), 6f451c105fdc4f048e1131b7cce29d52 (127.11.223.131:36025)
I20260621 07:56:50.833081 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898386c014342e1b9df7e7fa9b5930a" candidate_uuid: "2ceb5a700eed4f66a6010389bdc298af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbeedbfef8624b18b2d5340146daa883"
I20260621 07:56:50.833129 17188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898386c014342e1b9df7e7fa9b5930a" candidate_uuid: "2ceb5a700eed4f66a6010389bdc298af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f451c105fdc4f048e1131b7cce29d52"
I20260621 07:56:50.833220 17188 raft_consensus.cc:3037] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:50.833213 17057 raft_consensus.cc:3037] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:50.834478 17057 raft_consensus.cc:2454] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ceb5a700eed4f66a6010389bdc298af in term 1.
I20260621 07:56:50.834478 17188 raft_consensus.cc:2454] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ceb5a700eed4f66a6010389bdc298af in term 1.
I20260621 07:56:50.834738 16861 leader_election.cc:302] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ceb5a700eed4f66a6010389bdc298af, dbeedbfef8624b18b2d5340146daa883; no voters:
I20260621 07:56:50.834906 17257 raft_consensus.cc:2781] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:50.835155 17257 raft_consensus.cc:686] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [term 1 LEADER]: Becoming Leader. State: Replica: 2ceb5a700eed4f66a6010389bdc298af, State: Running, Role: LEADER
I20260621 07:56:50.835508 17257 consensus_queue.cc:237] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } }
I20260621 07:56:50.838536 16783 catalog_manager.cc:5495] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ceb5a700eed4f66a6010389bdc298af (127.11.223.129). New cstate: current_term: 1 leader_uuid: "2ceb5a700eed4f66a6010389bdc298af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ceb5a700eed4f66a6010389bdc298af" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260621 07:56:50.847146 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.845666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.11.223.129:45591)
0621 07:56:50.845782 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:50.845787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:50.845823 (+ 36us) server_negotiation.cc:407] Connection header received
0621 07:56:50.846013 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:50.846015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:50.846067 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:50.846155 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:50.846743 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:50.846746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:50.846749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:50.846759 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:50.846777 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:50.846792 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:50.846880 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:50.846989 (+ 109us) server_negotiation.cc:299] Negotiation successful
0621 07:56:50.847023 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:50.852406 17057 raft_consensus.cc:1261] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 1 FOLLOWER]: Refusing update from remote peer 2ceb5a700eed4f66a6010389bdc298af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:50.852406 17188 raft_consensus.cc:1261] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 1 FOLLOWER]: Refusing update from remote peer 2ceb5a700eed4f66a6010389bdc298af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:50.853119 17261 consensus_queue.cc:1035] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f451c105fdc4f048e1131b7cce29d52" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:50.853324 17257 consensus_queue.cc:1035] T 5898386c014342e1b9df7e7fa9b5930a P 2ceb5a700eed4f66a6010389bdc298af [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbeedbfef8624b18b2d5340146daa883" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:50.869179 16784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:50.869372 16784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:50.871608 16784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e544535a7f84c0d8cd904f81819cc61: Sending DeleteTablet for 3 replicas of tablet 5898386c014342e1b9df7e7fa9b5930a
I20260621 07:56:50.872293 17168 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898386c014342e1b9df7e7fa9b5930a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:50 UTC) from {username='slave'} at 127.0.0.1:48876
I20260621 07:56:50.872419 17037 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898386c014342e1b9df7e7fa9b5930a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:50 UTC) from {username='slave'} at 127.0.0.1:35436
I20260621 07:56:50.872750 17274 tablet_replica.cc:330] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: stopping tablet replica
I20260621 07:56:50.873087 17274 raft_consensus.cc:2229] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:50.873134 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16844
I20260621 07:56:50.873116 16906 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898386c014342e1b9df7e7fa9b5930a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:50 UTC) from {username='slave'} at 127.0.0.1:38388
I20260621 07:56:50.873775 17275 tablet_replica.cc:330] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: stopping tablet replica
I20260621 07:56:50.874106 17275 raft_consensus.cc:2229] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:50.874274 17274 raft_consensus.cc:2258] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:50.874408 17275 raft_consensus.cc:2258] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:50.875854 17275 ts_tablet_manager.cc:1905] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:50.876500 17274 ts_tablet_manager.cc:1905] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:50.878491 17274 ts_tablet_manager.cc:1918] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:50.878602 17274 log.cc:1196] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/5898386c014342e1b9df7e7fa9b5930a
I20260621 07:56:50.878887 17274 ts_tablet_manager.cc:1939] T 5898386c014342e1b9df7e7fa9b5930a P 6f451c105fdc4f048e1131b7cce29d52: Deleting consensus metadata
I20260621 07:56:50.879307 17275 ts_tablet_manager.cc:1918] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:50.879402 17275 log.cc:1196] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/5898386c014342e1b9df7e7fa9b5930a
I20260621 07:56:50.879735 17275 ts_tablet_manager.cc:1939] T 5898386c014342e1b9df7e7fa9b5930a P dbeedbfef8624b18b2d5340146daa883: Deleting consensus metadata
I20260621 07:56:50.880105 16769 catalog_manager.cc:4841] TS 6f451c105fdc4f048e1131b7cce29d52 (127.11.223.131:36025): tablet 5898386c014342e1b9df7e7fa9b5930a (table test-table [id=72ceba5cb1c54b48a82d4d00cc0f00be]) successfully deleted
I20260621 07:56:50.880839 16770 catalog_manager.cc:4841] TS dbeedbfef8624b18b2d5340146daa883 (127.11.223.130:39979): tablet 5898386c014342e1b9df7e7fa9b5930a (table test-table [id=72ceba5cb1c54b48a82d4d00cc0f00be]) successfully deleted
W20260621 07:56:50.881072 16770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:45591 (error 108)
I20260621 07:56:50.881110 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16975
I20260621 07:56:50.882397 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:50.882057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:45591 (local address 127.0.0.1:38402)
0621 07:56:50.882215 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:50.882293 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 07:56:50.882562 16770 catalog_manager.cc:4584] TS 2ceb5a700eed4f66a6010389bdc298af (127.11.223.129:45591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5898386c014342e1b9df7e7fa9b5930a: Network error: Client connection negotiation failed: client connection to 127.11.223.129:45591: connect: Connection refused (error 111)
I20260621 07:56:50.886494 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17106
I20260621 07:56:50.891698 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 16753
2026-06-21T07:56:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (894 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260621 07:56:50.912904 12158 mini_oidc.cc:152] Starting JWKS server
I20260621 07:56:50.914366 12158 webserver.cc:460] Webserver started at https://127.0.0.1:35425/ using document root <none> and password file <none>
I20260621 07:56:50.914515 12158 mini_oidc.cc:189] Starting OIDC Discovery server
I20260621 07:56:50.914729 12158 webserver.cc:460] Webserver started at http://127.0.0.1:33179/ using document root <none> and password file <none>
2026-06-21T07:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:50Z Disabled control of system clock
I20260621 07:56:50.923561 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43777
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43777
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35425/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/testchainca.cert with env {}
W20260621 07:56:51.029670 17282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:51.029922 17282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:51.029959 17282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:51.034000 17282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:51.034049 17282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:51.034065 17282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:51.034101 17282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:51.034130 17282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 07:56:51.034149 17282 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 07:56:51.034189 17282 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35425/jwks.json
I20260621 07:56:51.039276 17282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35425/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:51.040221 17282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:51.041229 17282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:51.049149 17290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.049183 17288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.049180 17287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:51.049427 17282 server_base.cc:1034] running on GCE node
I20260621 07:56:51.050040 17282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:51.051000 17282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:51.052151 17282 hybrid_clock.cc:648] HybridClock initialized: now 1782028611052134 us; error 29 us; skew 500 ppm
I20260621 07:56:51.053802 17282 webserver.cc:460] Webserver started at http://127.11.223.190:35793/ using document root <none> and password file <none>
I20260621 07:56:51.054275 17282 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:51.054328 17282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:51.054478 17282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:51.056219 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "a7a240991f914ae08724a4ed0409af91"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "3249d6509ef98969bf58250630f397f2"
server_key_iv: "d1de8ac18d1f0cac86586fd6f225ba4b"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.056766 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "a7a240991f914ae08724a4ed0409af91"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "3249d6509ef98969bf58250630f397f2"
server_key_iv: "d1de8ac18d1f0cac86586fd6f225ba4b"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.059893 17282 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:51.062098 17296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:51.063169 17282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:51.063259 17282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "a7a240991f914ae08724a4ed0409af91"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "3249d6509ef98969bf58250630f397f2"
server_key_iv: "d1de8ac18d1f0cac86586fd6f225ba4b"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.063335 17282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:51.084522 17282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:51.085099 17282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:51.085240 17282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:51.092664 17282 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:43777
I20260621 07:56:51.092696 17348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:43777 every 8 connection(s)
I20260621 07:56:51.093973 17282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:51.097904 17349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:51.098982 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17282
I20260621 07:56:51.099108 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:51.099416 12158 external_mini_cluster.cc:1428] Setting key 1863fc7ab4d3a34395720f2c1ad9bdd8
I20260621 07:56:51.103765 17349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91: Bootstrap starting.
I20260621 07:56:51.105926 17349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:51.106814 17349 log.cc:824] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:51.108124 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.100734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.11.223.190:43777)
0621 07:56:51.101334 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.101347 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.101388 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:56:51.101976 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.101991 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.102345 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.102665 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:51.103072 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.104050 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.105001 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.105441 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.105610 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.106463 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:51.106475 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:51.106487 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:51.106844 (+ 357us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:51.106927 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:51.106956 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:51.107066 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:51.107210 (+ 144us) server_negotiation.cc:299] Negotiation successful
0621 07:56:51.107461 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":138,"threads_started":1}
I20260621 07:56:51.108752 17349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91: No bootstrap required, opened a new log
I20260621 07:56:51.111003 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:56:51.113030 17349 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a240991f914ae08724a4ed0409af91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43777 } }
I20260621 07:56:51.113240 17349 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:51.113281 17349 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a240991f914ae08724a4ed0409af91, State: Initialized, Role: FOLLOWER
I20260621 07:56:51.113613 17349 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a240991f914ae08724a4ed0409af91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43777 } }
I20260621 07:56:51.113713 17349 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:51.113754 17349 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:51.113827 17349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:51.114673 17349 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a240991f914ae08724a4ed0409af91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43777 } }
I20260621 07:56:51.114953 17349 leader_election.cc:302] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7a240991f914ae08724a4ed0409af91; no voters:
I20260621 07:56:51.115298 17349 leader_election.cc:288] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:51.115396 17354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:51.115635 17354 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [term 1 LEADER]: Becoming Leader. State: Replica: a7a240991f914ae08724a4ed0409af91, State: Running, Role: LEADER
I20260621 07:56:51.115957 17354 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a240991f914ae08724a4ed0409af91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43777 } }
I20260621 07:56:51.116540 17349 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:51.117585 17356 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7a240991f914ae08724a4ed0409af91. Latest consensus state: current_term: 1 leader_uuid: "a7a240991f914ae08724a4ed0409af91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a240991f914ae08724a4ed0409af91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43777 } } }
I20260621 07:56:51.117712 17356 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.117988 17355 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7a240991f914ae08724a4ed0409af91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a240991f914ae08724a4ed0409af91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43777 } } }
I20260621 07:56:51.118062 17355 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.118865 17363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:51.121429 17363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:51.126434 17363 catalog_manager.cc:1347] Generated new cluster ID: b07ed6dc87f040e5823fd49ad905115e
I20260621 07:56:51.126498 17363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:51.140714 17363 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:51.141909 17363 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:51.147751 17363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7a240991f914ae08724a4ed0409af91: Generated new TSK 0
I20260621 07:56:51.148486 17363 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260621 07:56:51.175825 17379 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260621 07:56:51.175957 17379 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260621 07:56:51.176538 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.159155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.11.223.190:43777)
0621 07:56:51.159291 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.159296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.159475 (+ 179us) server_negotiation.cc:407] Connection header received
0621 07:56:51.159657 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.159660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.159727 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.159843 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0621 07:56:51.160085 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.160961 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.161816 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.162171 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.162270 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.162575 (+ 305us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 07:56:51.176107 (+ 13532us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0621 07:56:51.176400 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260621 07:56:51.176618 17352 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260621 07:56:51.177701 17378 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:51.159037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43777 (local address 127.0.0.1:45946)
0621 07:56:51.159394 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:51.159406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:51.159509 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:51.159863 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:51.159866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:51.159873 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0621 07:56:51.159980 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:51.159985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.161081 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.161084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:51.161700 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:51.161707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.162281 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.162284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:51.162348 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.162398 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 07:56:51.176370 (+ 13972us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0621 07:56:51.176582 (+ 212us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.223.190:43777: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":104,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.223.190:43777: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260621 07:56:51.178591 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17282
2026-06-21T07:56:51Z chronyd exiting
I20260621 07:56:51.215757 12158 test_util.cc:175] -----------------------------------------------
I20260621 07:56:51.215837 12158 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1782028587408853-12158-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (304 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260621 07:56:51.216944 12158 mini_oidc.cc:152] Starting JWKS server
I20260621 07:56:51.217796 12158 webserver.cc:460] Webserver started at https://127.0.0.1:43887/ using document root <none> and password file <none>
I20260621 07:56:51.217912 12158 mini_oidc.cc:189] Starting OIDC Discovery server
I20260621 07:56:51.218194 12158 webserver.cc:460] Webserver started at http://127.0.0.1:42351/ using document root <none> and password file <none>
2026-06-21T07:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:51Z Disabled control of system clock
I20260621 07:56:51.226701 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32801
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:32801
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43887/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/testchainca.cert with env {}
W20260621 07:56:51.332051 17385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:51.332283 17385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:51.332398 17385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:51.336290 17385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:51.336354 17385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:51.336369 17385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:51.336409 17385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:51.336439 17385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 07:56:51.336467 17385 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 07:56:51.336503 17385 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43887/jwks.json
I20260621 07:56:51.341634 17385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:32801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43887/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:51.342619 17385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:51.343549 17385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:51.351590 17391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.351611 17393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.351619 17390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:51.351954 17385 server_base.cc:1034] running on GCE node
I20260621 07:56:51.352542 17385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:51.353544 17385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:51.354702 17385 hybrid_clock.cc:648] HybridClock initialized: now 1782028611354687 us; error 28 us; skew 500 ppm
I20260621 07:56:51.356415 17385 webserver.cc:460] Webserver started at http://127.11.223.190:35803/ using document root <none> and password file <none>
I20260621 07:56:51.356943 17385 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:51.357003 17385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:51.357189 17385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:51.359037 17385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "094cda9a8e9e4bfa8163d9d9d23927f3"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "5fd3ec2152bedfee7c41e6a32c8e9d86"
server_key_iv: "cf5f6cdb587d24f458c612acd81b77eb"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.359578 17385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "094cda9a8e9e4bfa8163d9d9d23927f3"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "5fd3ec2152bedfee7c41e6a32c8e9d86"
server_key_iv: "cf5f6cdb587d24f458c612acd81b77eb"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.362923 17385 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260621 07:56:51.365196 17399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:51.366361 17385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:51.366494 17385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "094cda9a8e9e4bfa8163d9d9d23927f3"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "5fd3ec2152bedfee7c41e6a32c8e9d86"
server_key_iv: "cf5f6cdb587d24f458c612acd81b77eb"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.366598 17385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:51.379128 17385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:51.379719 17385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:51.379915 17385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:51.386435 17385 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:32801
I20260621 07:56:51.386442 17451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:32801 every 8 connection(s)
I20260621 07:56:51.387276 17385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:51.390273 17452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:51.391712 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17385
I20260621 07:56:51.391841 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:51.392127 12158 external_mini_cluster.cc:1428] Setting key 75f9c60b7894f5c4566bcc8906a4b7ac
I20260621 07:56:51.396067 17452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3: Bootstrap starting.
I20260621 07:56:51.398193 17452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:51.399008 17452 log.cc:824] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:51.401288 17452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3: No bootstrap required, opened a new log
I20260621 07:56:51.401918 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.393451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.11.223.190:32801)
0621 07:56:51.394030 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.394043 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.394075 (+ 32us) server_negotiation.cc:407] Connection header received
0621 07:56:51.394679 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.394694 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.395045 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.395361 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:51.395773 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.396843 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.397734 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.398192 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.398382 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.399866 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:51.399901 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:51.399917 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:51.400437 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:51.400499 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:51.400524 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:51.400623 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:51.400734 (+ 111us) server_negotiation.cc:299] Negotiation successful
0621 07:56:51.400979 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":122,"threads_started":1}
I20260621 07:56:51.404814 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:56:51.405445 17452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32801 } }
I20260621 07:56:51.405731 17452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:51.405799 17452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094cda9a8e9e4bfa8163d9d9d23927f3, State: Initialized, Role: FOLLOWER
I20260621 07:56:51.406279 17452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32801 } }
I20260621 07:56:51.406437 17452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:51.406522 17452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:51.406622 17452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:51.407477 17452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32801 } }
I20260621 07:56:51.407793 17452 leader_election.cc:302] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 094cda9a8e9e4bfa8163d9d9d23927f3; no voters:
I20260621 07:56:51.408074 17452 leader_election.cc:288] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:51.408191 17457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:51.408475 17457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [term 1 LEADER]: Becoming Leader. State: Replica: 094cda9a8e9e4bfa8163d9d9d23927f3, State: Running, Role: LEADER
I20260621 07:56:51.408842 17457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32801 } }
I20260621 07:56:51.409276 17452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:51.410477 17458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32801 } } }
I20260621 07:56:51.410457 17459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 094cda9a8e9e4bfa8163d9d9d23927f3. Latest consensus state: current_term: 1 leader_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094cda9a8e9e4bfa8163d9d9d23927f3" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32801 } } }
I20260621 07:56:51.410635 17458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.410638 17459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.411161 17466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:51.413825 17466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:51.418988 17466 catalog_manager.cc:1347] Generated new cluster ID: d79e27b11dcf4bca869d86dcb5890ce7
I20260621 07:56:51.419047 17466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:51.424753 17466 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:51.425802 17466 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:51.437711 17466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 094cda9a8e9e4bfa8163d9d9d23927f3: Generated new TSK 0
I20260621 07:56:51.438412 17466 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260621 07:56:51.468048 17482 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260621 07:56:51.468125 17482 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260621 07:56:51.468704 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.453667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.11.223.190:32801)
0621 07:56:51.453810 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.453817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.453995 (+ 178us) server_negotiation.cc:407] Connection header received
0621 07:56:51.454138 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.454140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.454184 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.454270 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0621 07:56:51.454541 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.455087 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.456278 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.456539 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.456614 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.456932 (+ 318us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 07:56:51.468424 (+ 11492us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0621 07:56:51.468609 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37580: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260621 07:56:51.468784 17455 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37580: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260621 07:56:51.468894 17481 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:51.453596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32801 (local address 127.0.0.1:37580)
0621 07:56:51.453895 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:51.453911 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:51.454023 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:51.454289 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:51.454298 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:51.454306 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0621 07:56:51.454437 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:51.454444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.455211 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.455215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:51.456161 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:51.456171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.456642 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.456645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:51.456731 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.456759 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 07:56:51.468573 (+ 11814us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0621 07:56:51.468728 (+ 155us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.223.190:32801: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
I20260621 07:56:51.469466 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17385
2026-06-21T07:56:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (291 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260621 07:56:51.508162 12158 mini_oidc.cc:152] Starting JWKS server
I20260621 07:56:51.509021 12158 webserver.cc:460] Webserver started at https://127.0.0.1:45201/ using document root <none> and password file <none>
I20260621 07:56:51.509097 12158 mini_oidc.cc:189] Starting OIDC Discovery server
I20260621 07:56:51.509378 12158 webserver.cc:460] Webserver started at http://127.0.0.1:41565/ using document root <none> and password file <none>
2026-06-21T07:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:51Z Disabled control of system clock
I20260621 07:56:51.517764 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:40191
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:40191
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260621 07:56:51.624643 17488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:51.624878 17488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:51.624914 17488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:51.628619 17488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:51.628665 17488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:51.628681 17488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:51.628697 17488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:51.628711 17488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 07:56:51.628726 17488 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 07:56:51.628739 17488 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45201/jwks.json
I20260621 07:56:51.633402 17488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:40191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:40191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:51.634268 17488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:51.635562 17488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:51.643332 17494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.643388 17493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:51.643494 17488 server_base.cc:1034] running on GCE node
W20260621 07:56:51.643695 17496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:51.644243 17488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:51.645287 17488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:51.646441 17488 hybrid_clock.cc:648] HybridClock initialized: now 1782028611646424 us; error 30 us; skew 500 ppm
I20260621 07:56:51.648099 17488 webserver.cc:460] Webserver started at http://127.11.223.190:44105/ using document root <none> and password file <none>
I20260621 07:56:51.648672 17488 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:51.648729 17488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:51.648950 17488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:51.650892 17488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "59f9129e969a48c28edb98a555fa5cd8"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "d03d72a30aa3b460bcffdb644110f70e"
server_key_iv: "29d20810e27e0a73a8695ba7f7c49966"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.651455 17488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "59f9129e969a48c28edb98a555fa5cd8"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "d03d72a30aa3b460bcffdb644110f70e"
server_key_iv: "29d20810e27e0a73a8695ba7f7c49966"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.654759 17488 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260621 07:56:51.656967 17502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:51.658056 17488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:51.658190 17488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "59f9129e969a48c28edb98a555fa5cd8"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "d03d72a30aa3b460bcffdb644110f70e"
server_key_iv: "29d20810e27e0a73a8695ba7f7c49966"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.658304 17488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:51.675575 17488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:51.676163 17488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:51.676403 17488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:51.682749 17488 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:40191
I20260621 07:56:51.682767 17554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:40191 every 8 connection(s)
I20260621 07:56:51.683647 17488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:51.686393 17555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:51.691538 17555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8: Bootstrap starting.
I20260621 07:56:51.692826 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17488
I20260621 07:56:51.692946 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:51.693231 12158 external_mini_cluster.cc:1428] Setting key fa17588920899e4a96d5f14e6b3add24
I20260621 07:56:51.694233 17555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:51.695007 17555 log.cc:824] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:51.696882 17555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8: No bootstrap required, opened a new log
I20260621 07:56:51.700412 17555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f9129e969a48c28edb98a555fa5cd8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40191 } }
I20260621 07:56:51.700605 17555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:51.700673 17555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f9129e969a48c28edb98a555fa5cd8, State: Initialized, Role: FOLLOWER
I20260621 07:56:51.701152 17555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f9129e969a48c28edb98a555fa5cd8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40191 } }
I20260621 07:56:51.701304 17555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:51.701390 17555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:51.701483 17555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:51.702072 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.694296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.11.223.190:40191)
0621 07:56:51.694899 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.694913 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.694943 (+ 30us) server_negotiation.cc:407] Connection header received
0621 07:56:51.695688 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.695711 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.696036 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.696373 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:51.696759 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.697716 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.698636 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.699101 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.699274 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.700112 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:51.700117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:51.700129 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:51.700566 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:51.700645 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:51.700692 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:51.700809 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:51.701006 (+ 197us) server_negotiation.cc:299] Negotiation successful
0621 07:56:51.701264 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":127,"threads_started":1}
I20260621 07:56:51.702389 17555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f9129e969a48c28edb98a555fa5cd8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40191 } }
I20260621 07:56:51.702724 17555 leader_election.cc:302] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59f9129e969a48c28edb98a555fa5cd8; no voters:
I20260621 07:56:51.703006 17555 leader_election.cc:288] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:51.703101 17560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:51.703287 17560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [term 1 LEADER]: Becoming Leader. State: Replica: 59f9129e969a48c28edb98a555fa5cd8, State: Running, Role: LEADER
I20260621 07:56:51.703580 17560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f9129e969a48c28edb98a555fa5cd8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40191 } }
I20260621 07:56:51.704171 17555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:51.704722 17562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59f9129e969a48c28edb98a555fa5cd8. Latest consensus state: current_term: 1 leader_uuid: "59f9129e969a48c28edb98a555fa5cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f9129e969a48c28edb98a555fa5cd8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40191 } } }
I20260621 07:56:51.704948 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:56:51.704963 17562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.705116 17561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59f9129e969a48c28edb98a555fa5cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f9129e969a48c28edb98a555fa5cd8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40191 } } }
I20260621 07:56:51.705193 17561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8 [sys.catalog]: This master's current role is: LEADER
W20260621 07:56:51.708611 17575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:56:51.708694 17575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:56:51.708768 17568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:51.709445 17568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:51.714515 17568 catalog_manager.cc:1347] Generated new cluster ID: 69946a752efc4473b344af39d91fb255
I20260621 07:56:51.714573 17568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:51.726275 17568 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:51.727376 17568 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:51.733103 17568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59f9129e969a48c28edb98a555fa5cd8: Generated new TSK 0
I20260621 07:56:51.733788 17568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:56:51.752224 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.746031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.11.223.190:40191)
0621 07:56:51.746164 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.746168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.746310 (+ 142us) server_negotiation.cc:407] Connection header received
0621 07:56:51.746562 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.746566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.746627 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.746753 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0621 07:56:51.747142 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.747940 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.749277 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.749848 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.750032 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.750550 (+ 518us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 07:56:51.751857 (+ 1307us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0621 07:56:51.752086 (+ 229us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54192: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":55}
W20260621 07:56:51.752283 17558 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54192: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260621 07:56:51.754861 17584 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:51.745935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:40191 (local address 127.0.0.1:54192)
0621 07:56:51.746227 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:51.746241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:51.746382 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:51.746867 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:51.746872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:51.746881 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0621 07:56:51.746992 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:51.746998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.748099 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.748103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:51.749122 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:51.749134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.750201 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.750208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:51.750325 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.750361 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 07:56:51.752042 (+ 1681us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0621 07:56:51.754685 (+ 2643us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.223.190:40191: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":2408,"thread_start_us":110,"threads_started":1}
I20260621 07:56:51.755331 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17488
E20260621 07:56:51.764045 17585 webserver.cc:533] Webserver: sslize failed:
E20260621 07:56:51.764102 17585 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-21T07:56:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (285 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-21T07:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:51Z Disabled control of system clock
I20260621 07:56:51.802407 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46619
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:46619
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/testchainca.cert with env {}
W20260621 07:56:51.905627 17589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:51.905864 17589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:51.905900 17589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:51.909746 17589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:51.909795 17589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:51.909811 17589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:51.909850 17589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:51.909883 17589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260621 07:56:51.909910 17589 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260621 07:56:51.909940 17589 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260621 07:56:51.914963 17589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:46619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:51.915863 17589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:51.916826 17589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:51.924449 17594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.924445 17595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:51.924504 17597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:51.924654 17589 server_base.cc:1034] running on GCE node
I20260621 07:56:51.925261 17589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:51.926167 17589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:51.927335 17589 hybrid_clock.cc:648] HybridClock initialized: now 1782028611927301 us; error 48 us; skew 500 ppm
I20260621 07:56:51.929023 17589 webserver.cc:460] Webserver started at http://127.11.223.190:34421/ using document root <none> and password file <none>
I20260621 07:56:51.929557 17589 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:51.929617 17589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:51.929857 17589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:51.931659 17589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "6b1fba8b13614233a04b9747f5fc3775"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "b056d7e5ccf40101d03bde2f0e2d4a5b"
server_key_iv: "76d2160cdccc986e80323fe35713ad82"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.932178 17589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "6b1fba8b13614233a04b9747f5fc3775"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "b056d7e5ccf40101d03bde2f0e2d4a5b"
server_key_iv: "76d2160cdccc986e80323fe35713ad82"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.935433 17589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260621 07:56:51.937670 17603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:51.938819 17589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260621 07:56:51.938939 17589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "6b1fba8b13614233a04b9747f5fc3775"
format_stamp: "Formatted at 2026-06-21 07:56:51 on dist-test-slave-0pn3"
server_key: "b056d7e5ccf40101d03bde2f0e2d4a5b"
server_key_iv: "76d2160cdccc986e80323fe35713ad82"
server_key_version: "encryptionkey@0"
I20260621 07:56:51.939033 17589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:51.958170 17589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:51.958745 17589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:51.958921 17589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:51.965377 17589 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:46619
I20260621 07:56:51.965387 17655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:46619 every 8 connection(s)
I20260621 07:56:51.966337 17589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:51.967623 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17589
I20260621 07:56:51.967742 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:51.967993 12158 external_mini_cluster.cc:1428] Setting key 9a7cfdcfe6de2b2bfa11f40524076071
I20260621 07:56:51.969280 17656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:51.974835 17656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775: Bootstrap starting.
I20260621 07:56:51.976814 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:51.969183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.11.223.190:46619)
0621 07:56:51.969790 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:51.969803 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:51.969836 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:56:51.970414 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:51.970428 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:51.970783 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:51.971096 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:51.971574 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.972916 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.973796 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:51.974223 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:51.974393 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:51.975212 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:51.975217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:51.975227 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:56:51.975566 (+ 339us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:51.975615 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:56:51.975633 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:51.975734 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:51.975893 (+ 159us) server_negotiation.cc:299] Negotiation successful
0621 07:56:51.976120 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":128,"threads_started":1}
I20260621 07:56:51.976948 17656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:51.977599 17656 log.cc:824] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:51.979481 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:56:51.979489 17656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775: No bootstrap required, opened a new log
I20260621 07:56:51.982810 17656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fba8b13614233a04b9747f5fc3775" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46619 } }
I20260621 07:56:51.983031 17656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:51.983105 17656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1fba8b13614233a04b9747f5fc3775, State: Initialized, Role: FOLLOWER
I20260621 07:56:51.983474 17656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fba8b13614233a04b9747f5fc3775" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46619 } }
I20260621 07:56:51.983620 17656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:51.983700 17656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:51.983807 17656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:51.984704 17656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fba8b13614233a04b9747f5fc3775" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46619 } }
I20260621 07:56:51.985006 17656 leader_election.cc:302] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b1fba8b13614233a04b9747f5fc3775; no voters:
I20260621 07:56:51.985270 17656 leader_election.cc:288] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:51.985366 17661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:51.985620 17661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [term 1 LEADER]: Becoming Leader. State: Replica: 6b1fba8b13614233a04b9747f5fc3775, State: Running, Role: LEADER
I20260621 07:56:51.985913 17661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fba8b13614233a04b9747f5fc3775" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46619 } }
I20260621 07:56:51.986405 17656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:51.986873 17663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b1fba8b13614233a04b9747f5fc3775. Latest consensus state: current_term: 1 leader_uuid: "6b1fba8b13614233a04b9747f5fc3775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fba8b13614233a04b9747f5fc3775" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46619 } } }
I20260621 07:56:51.987043 17663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.987262 17662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b1fba8b13614233a04b9747f5fc3775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fba8b13614233a04b9747f5fc3775" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46619 } } }
I20260621 07:56:51.987355 17662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:51.987578 17669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:51.990279 17669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:51.995508 17669 catalog_manager.cc:1347] Generated new cluster ID: 12e3722c1cb447dba680aea614c81349
I20260621 07:56:51.995575 17669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:52.001029 17669 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:52.001773 17669 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:52.006248 17669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b1fba8b13614233a04b9747f5fc3775: Generated new TSK 0
I20260621 07:56:52.006788 17669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:56:52.039708 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:52.020827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.11.223.190:46619)
0621 07:56:52.020939 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:52.020944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:52.021098 (+ 154us) server_negotiation.cc:407] Connection header received
0621 07:56:52.021235 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:52.021238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:52.021280 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:52.021358 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0621 07:56:52.021602 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:52.022086 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:52.023344 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:52.023546 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:52.023618 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:52.023975 (+ 357us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0621 07:56:52.039393 (+ 15418us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0621 07:56:52.039603 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36874: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":45}
W20260621 07:56:52.039896 17659 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36874: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260621 07:56:52.039924 17685 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:52.020681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46619 (local address 127.0.0.1:36874)
0621 07:56:52.021005 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:52.021020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:52.021127 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:52.021386 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:52.021389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:52.021398 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0621 07:56:52.021500 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:52.021506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:52.022212 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:52.022215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:52.023212 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:52.023221 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:52.023661 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:52.023665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:52.023757 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:52.023783 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0621 07:56:52.039574 (+ 15791us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0621 07:56:52.039745 (+ 171us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.223.190:46619: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":123,"threads_started":1}
I20260621 07:56:52.040685 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17589
2026-06-21T07:56:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (264 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (112 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (115 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:52 dist-test-slave-0pn3 krb5kdc[17697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:52 dist-test-slave-0pn3 krb5kdc[17697](info): set up 2 sockets
Jun 21 07:56:52 dist-test-slave-0pn3 krb5kdc[17697](info): commencing operation
krb5kdc: starting...
W20260621 07:56:53.280956 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:53 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:56:53.391127 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33709
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:33709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:53.493815 17710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:53.494059 17710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:53.494153 17710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:53.497982 17710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:53.498050 17710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:53.498085 17710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:53.498139 17710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:53.498188 17710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:53.503134 17710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:33709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:53.504069 17710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:53.505040 17710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:53.513044 17716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:53.513113 17715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:53.513063 17718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:53.513335 17710 server_base.cc:1034] running on GCE node
I20260621 07:56:53.513936 17710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:53.514906 17710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:53.516090 17710 hybrid_clock.cc:648] HybridClock initialized: now 1782028613516059 us; error 41 us; skew 500 ppm
Jun 21 07:56:53 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:53.518584 17710 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:56:53.519413 17710 webserver.cc:460] Webserver started at http://127.11.223.190:34191/ using document root <none> and password file <none>
I20260621 07:56:53.519941 17710 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:53.520030 17710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:53.520246 17710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:53.522110 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "1ef76f13764b469fb3412cc3f78b5f9c"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "d0a2920f4e6e35e1d32a6ddd9062861d"
server_key_iv: "7f83797ae05715d519f3ef8fb0a73fb1"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.522635 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "1ef76f13764b469fb3412cc3f78b5f9c"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "d0a2920f4e6e35e1d32a6ddd9062861d"
server_key_iv: "7f83797ae05715d519f3ef8fb0a73fb1"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.526104 17710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:53.528182 17725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:53.529292 17710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:53.529452 17710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "1ef76f13764b469fb3412cc3f78b5f9c"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "d0a2920f4e6e35e1d32a6ddd9062861d"
server_key_iv: "7f83797ae05715d519f3ef8fb0a73fb1"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.529558 17710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:53.540407 17710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:53.542841 17710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:53.543037 17710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:53.549749 17710 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:33709
I20260621 07:56:53.549765 17777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:33709 every 8 connection(s)
I20260621 07:56:53.550611 17710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:53.553244 17778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:53.555922 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17710
I20260621 07:56:53.556069 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:53.556371 12158 external_mini_cluster.cc:1428] Setting key fa88b82564441fcbf90047f7ba48ac37
I20260621 07:56:53.558471 17778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c: Bootstrap starting.
I20260621 07:56:53.560896 17778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:53.561687 17778 log.cc:824] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:53.563629 17778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c: No bootstrap required, opened a new log
Jun 21 07:56:53 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:53.566649 17778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33709 } }
I20260621 07:56:53.566905 17778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:53.566995 17778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef76f13764b469fb3412cc3f78b5f9c, State: Initialized, Role: FOLLOWER
I20260621 07:56:53.567436 17778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33709 } }
I20260621 07:56:53.567576 17778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:53.567641 17778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:53.567739 17778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:53.568641 17778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33709 } }
I20260621 07:56:53.568946 17778 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef76f13764b469fb3412cc3f78b5f9c; no voters:
I20260621 07:56:53.569306 17778 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:53.569363 17783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:53.569664 17783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [term 1 LEADER]: Becoming Leader. State: Replica: 1ef76f13764b469fb3412cc3f78b5f9c, State: Running, Role: LEADER
I20260621 07:56:53.570003 17783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33709 } }
I20260621 07:56:53.570454 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:53.557676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.11.223.190:33709)
0621 07:56:53.558039 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:53.558053 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:53.558085 (+ 32us) server_negotiation.cc:407] Connection header received
0621 07:56:53.558688 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:53.558705 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:53.559011 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:53.559258 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:53.560174 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:53.561216 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:53.562293 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:53.562750 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:53.562873 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:53.565335 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:53.565339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:53.565350 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:53.565373 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:53.567477 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:53.567970 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:53.567975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:53.567981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:53.568117 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:53.568419 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:53.568423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:53.568424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:53.568844 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:53.569038 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:53.569298 (+ 260us) server_negotiation.cc:299] Negotiation successful
0621 07:56:53.569602 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":122,"threads_started":1}
I20260621 07:56:53.570627 17778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:53.571615 17784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33709 } } }
I20260621 07:56:53.571745 17784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:53.572060 17785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef76f13764b469fb3412cc3f78b5f9c. Latest consensus state: current_term: 1 leader_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef76f13764b469fb3412cc3f78b5f9c" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33709 } } }
I20260621 07:56:53.572114 17792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:53.572134 17785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:53.574682 17792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:53.579368 17792 catalog_manager.cc:1347] Generated new cluster ID: cebf93cd1eec43bf89f7f9739a1d2a80
I20260621 07:56:53.579454 17792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:53.585193 17792 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:53.586028 17792 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:53.601630 17792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef76f13764b469fb3412cc3f78b5f9c: Generated new TSK 0
I20260621 07:56:53.602212 17792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:56:53.631012 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33709
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:53.736202 17804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:53.736452 17804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:53.736490 17804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:53.740336 17804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:53.740381 17804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:53.740483 17804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:53.745168 17804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:53.746073 17804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:53.747007 17804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:53.755820 17812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:53.755995 17804 server_base.cc:1034] running on GCE node
W20260621 07:56:53.755820 17809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:53.755856 17810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:53.756475 17804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:53.757073 17804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:53.758225 17804 hybrid_clock.cc:648] HybridClock initialized: now 1782028613758198 us; error 38 us; skew 500 ppm
Jun 21 07:56:53 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:53.760936 17804 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:56:53.761716 17804 webserver.cc:460] Webserver started at http://127.11.223.129:34739/ using document root <none> and password file <none>
I20260621 07:56:53.762217 17804 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:53.762274 17804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:53.762531 17804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:53.764295 17804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "d30d938205804099b65255e3dc98a9ac"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "f08cbe5372ae88157ad52b5e0eace9c6"
server_key_iv: "57468a874ea784947ecffebaa4ae8ee9"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.764889 17804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "d30d938205804099b65255e3dc98a9ac"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "f08cbe5372ae88157ad52b5e0eace9c6"
server_key_iv: "57468a874ea784947ecffebaa4ae8ee9"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.768242 17804 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:53.770462 17819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:53.771683 17804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:53.771800 17804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "d30d938205804099b65255e3dc98a9ac"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "f08cbe5372ae88157ad52b5e0eace9c6"
server_key_iv: "57468a874ea784947ecffebaa4ae8ee9"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.771898 17804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:53.784641 17804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:53.787595 17804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:53.787777 17804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:53.788270 17804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:53.789034 17804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:53.789103 17804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:53.789170 17804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:53.789213 17804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:53.797833 17804 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:46633
I20260621 07:56:53.797860 17932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:46633 every 8 connection(s)
I20260621 07:56:53.798590 17804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:53.806468 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17804
I20260621 07:56:53.806598 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:53.806942 12158 external_mini_cluster.cc:1428] Setting key daa694795884a23f50ff01742486c3ec
Jun 21 07:56:53 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:53.811326 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:53.800191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:59817 (local address 127.11.223.190:33709)
0621 07:56:53.800338 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:53.800342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:53.801101 (+ 759us) server_negotiation.cc:407] Connection header received
0621 07:56:53.802373 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:53.802377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:53.802431 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:53.802528 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:53.803582 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:53.804111 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:53.805184 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:53.805402 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:53.805508 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:53.808044 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:53.808047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:53.808049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:53.808073 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:53.809351 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:53.809879 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:53.809884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:53.809886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:53.809964 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:53.810308 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:53.810311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:53.810312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:53.810481 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:53.810586 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:53.811043 (+ 457us) server_negotiation.cc:299] Negotiation successful
0621 07:56:53.811157 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:56:53.812103 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:53.800573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33709 (local address 127.11.223.129:59817)
0621 07:56:53.800973 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:53.801008 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:53.802166 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:53.802684 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:53.802691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:53.803000 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:53.803436 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:53.803448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:53.804235 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:53.804238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:53.805051 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:53.805059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:53.805538 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:53.805541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:53.805716 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:53.806256 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:53.806278 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:53.807887 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:53.809496 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:53.809502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:53.809514 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:53.809772 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:53.810062 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:53.810065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:53.810067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:53.810197 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:53.810584 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:53.810590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:53.810808 (+ 218us) client_negotiation.cc:769] Sending connection context
0621 07:56:53.810992 (+ 184us) client_negotiation.cc:240] Negotiation successful
0621 07:56:53.811275 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
I20260621 07:56:53.813385 17933 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33709
I20260621 07:56:53.813627 17933 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:53.814050 17933 heartbeater.cc:511] Master 127.11.223.190:33709 requested a full tablet report, sending...
I20260621 07:56:53.815649 17742 ts_manager.cc:194] Registered new tserver with Master: d30d938205804099b65255e3dc98a9ac (127.11.223.129:46633)
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:53.817014 17742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:59817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:56:53.832465 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33709
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:53.934674 17938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:53.934887 17938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:53.934926 17938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:53.938680 17938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:53.938728 17938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:53.938796 17938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:53.943635 17938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:53.944445 17938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:53.945228 17938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:53.953876 17946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:53.953855 17944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:53.953855 17943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:53.953938 17938 server_base.cc:1034] running on GCE node
I20260621 07:56:53.954452 17938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:53.955030 17938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:53.956207 17938 hybrid_clock.cc:648] HybridClock initialized: now 1782028613956193 us; error 22 us; skew 500 ppm
Jun 21 07:56:53 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:53.958819 17938 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:56:53.959614 17938 webserver.cc:460] Webserver started at http://127.11.223.130:39345/ using document root <none> and password file <none>
I20260621 07:56:53.960119 17938 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:53.960176 17938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:53.960464 17938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:53.962276 17938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "5194b9dca8dc4b0094383ee22194aba6"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "106775fffb0845495ce487e7742f746d"
server_key_iv: "7a84afb0be71f05b90992b1951729b4f"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.962795 17938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "5194b9dca8dc4b0094383ee22194aba6"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "106775fffb0845495ce487e7742f746d"
server_key_iv: "7a84afb0be71f05b90992b1951729b4f"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.966112 17938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:53.968434 17953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:53.969525 17938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:53.969661 17938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "5194b9dca8dc4b0094383ee22194aba6"
format_stamp: "Formatted at 2026-06-21 07:56:53 on dist-test-slave-0pn3"
server_key: "106775fffb0845495ce487e7742f746d"
server_key_iv: "7a84afb0be71f05b90992b1951729b4f"
server_key_version: "encryptionkey@0"
I20260621 07:56:53.969780 17938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:53.990512 17938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:53.992904 17938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:53.993085 17938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:53.993594 17938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:53.994324 17938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:53.994391 17938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:53.994458 17938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:53.994503 17938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:54.002590 17938 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:46307
I20260621 07:56:54.002612 18066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:46307 every 8 connection(s)
I20260621 07:56:54.003396 17938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:54.007629 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 17938
I20260621 07:56:54.007771 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:54.008033 12158 external_mini_cluster.cc:1428] Setting key 3a4d5fd5d1226f6376ceadcd5e055e47
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:54.017534 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.005093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:33569 (local address 127.11.223.190:33709)
0621 07:56:54.005222 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.005225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.005959 (+ 734us) server_negotiation.cc:407] Connection header received
0621 07:56:54.007229 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.007232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.007286 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.007389 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.008756 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.009311 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.011073 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.011307 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.011417 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.013772 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:54.013778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:54.013784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:54.013824 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:54.015534 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.016000 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.016003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.016005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.016096 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.016423 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.016428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.016431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.016662 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:54.016763 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:54.017230 (+ 467us) server_negotiation.cc:299] Negotiation successful
0621 07:56:54.017359 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:56:54.018131 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.005422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33709 (local address 127.11.223.130:33569)
0621 07:56:54.005819 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:54.005857 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:54.007035 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:54.007545 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:54.007551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:54.007965 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:54.008600 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:54.008623 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.009498 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.009502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:54.010949 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:54.010963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.011450 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.011454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:54.011618 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.012179 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:54.012194 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:54.013602 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:54.015646 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:54.015650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:54.015660 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:54.015904 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:54.016186 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:54.016188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:54.016190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:54.016304 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:54.016763 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:54.016767 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:54.016978 (+ 211us) client_negotiation.cc:769] Sending connection context
0621 07:56:54.017132 (+ 154us) client_negotiation.cc:240] Negotiation successful
0621 07:56:54.017353 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:56:54.019209 18067 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33709
I20260621 07:56:54.019490 18067 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:54.019942 18067 heartbeater.cc:511] Master 127.11.223.190:33709 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 07:56:54.021080 17742 ts_manager.cc:194] Registered new tserver with Master: 5194b9dca8dc4b0094383ee22194aba6 (127.11.223.130:46307)
I20260621 07:56:54.021919 17742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:33569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:56:54.034682 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33709
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:54.138170 18072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:54.138408 18072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:54.138494 18072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:54.142539 18072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:54.142616 18072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:54.142720 18072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:54.147547 18072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:54.148468 18072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:54.149394 18072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:54.158603 18077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:54.158609 18078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:54.158609 18080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:54.158849 18072 server_base.cc:1034] running on GCE node
I20260621 07:56:54.159206 18072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:54.159771 18072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:54.160957 18072 hybrid_clock.cc:648] HybridClock initialized: now 1782028614160941 us; error 40 us; skew 500 ppm
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.167870 18072 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:56:54.168761 18072 webserver.cc:460] Webserver started at http://127.11.223.131:35903/ using document root <none> and password file <none>
I20260621 07:56:54.169272 18072 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:54.169365 18072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:54.169593 18072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:54.171697 18072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "833e1b0bbd804962b13a84528c13b8e8"
format_stamp: "Formatted at 2026-06-21 07:56:54 on dist-test-slave-0pn3"
server_key: "3e23d0e22819e93ff22f93c1be9cbdcd"
server_key_iv: "9b5587868971ffccfea50bcdb4ec93e4"
server_key_version: "encryptionkey@0"
I20260621 07:56:54.172292 18072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "833e1b0bbd804962b13a84528c13b8e8"
format_stamp: "Formatted at 2026-06-21 07:56:54 on dist-test-slave-0pn3"
server_key: "3e23d0e22819e93ff22f93c1be9cbdcd"
server_key_iv: "9b5587868971ffccfea50bcdb4ec93e4"
server_key_version: "encryptionkey@0"
I20260621 07:56:54.175725 18072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:56:54.178000 18087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:54.179137 18072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260621 07:56:54.179270 18072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "833e1b0bbd804962b13a84528c13b8e8"
format_stamp: "Formatted at 2026-06-21 07:56:54 on dist-test-slave-0pn3"
server_key: "3e23d0e22819e93ff22f93c1be9cbdcd"
server_key_iv: "9b5587868971ffccfea50bcdb4ec93e4"
server_key_version: "encryptionkey@0"
I20260621 07:56:54.179368 18072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:54.198421 18072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:54.201007 18072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:54.201184 18072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:54.201669 18072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:54.202399 18072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:54.202466 18072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:54.202536 18072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:54.202590 18072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:54.211038 18072 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:44505
I20260621 07:56:54.211076 18200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:44505 every 8 connection(s)
I20260621 07:56:54.211851 18072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:54.220118 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18072
I20260621 07:56:54.220235 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:56:54.220546 12158 external_mini_cluster.cc:1428] Setting key 1409fac80233c315d805b9eb94b697e7
I20260621 07:56:54.224224 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.213489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:33263 (local address 127.11.223.190:33709)
0621 07:56:54.213607 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.213610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.214333 (+ 723us) server_negotiation.cc:407] Connection header received
0621 07:56:54.215594 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.215597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.215660 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.215734 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.216772 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.217273 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.218278 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.218494 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.218596 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.221203 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:54.221206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:54.221208 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:54.221235 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:54.222669 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.223100 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.223102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.223103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.223171 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.223437 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.223439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.223440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.223588 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:54.223657 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:54.224035 (+ 378us) server_negotiation.cc:299] Negotiation successful
0621 07:56:54.224118 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:56:54.224954 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.213801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33709 (local address 127.11.223.131:33263)
0621 07:56:54.214193 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:54.214229 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:54.215398 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:54.215888 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:54.215894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:54.216200 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:54.216631 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:54.216642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.217404 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.217406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:54.218160 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:54.218168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.218625 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.218628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:54.218779 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.219354 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:54.219368 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:54.221018 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:54.222770 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:54.222774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:54.222783 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:54.223011 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:54.223243 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:54.223245 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:54.223246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:54.223365 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:54.223667 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:54.223671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:54.223858 (+ 187us) client_negotiation.cc:769] Sending connection context
0621 07:56:54.223994 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 07:56:54.224195 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":112,"threads_started":1}
I20260621 07:56:54.225857 18201 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33709
I20260621 07:56:54.226054 18201 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:54.226491 18201 heartbeater.cc:511] Master 127.11.223.190:33709 requested a full tablet report, sending...
I20260621 07:56:54.227350 17742 ts_manager.cc:194] Registered new tserver with Master: 833e1b0bbd804962b13a84528c13b8e8 (127.11.223.131:44505)
I20260621 07:56:54.227872 17742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:33263
I20260621 07:56:54.233937 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/test-admin.keytab.
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.248250 12158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260621 07:56:54.250849 18208 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:54.249472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.11.223.129:46633)
0621 07:56:54.249831 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.249841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.249859 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:56:54.249900 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.249904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.250050 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.250192 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.250671 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56504: BlockingRecv error: recv got EOF from 127.0.0.1:56504 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":129,"threads_started":1}
W20260621 07:56:54.250960 12158 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc' of type FILE
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.251907 12158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.252791 12158 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:54.261646 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.253539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.11.223.129:46633)
0621 07:56:54.253657 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.253661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.253752 (+ 91us) server_negotiation.cc:407] Connection header received
0621 07:56:54.253885 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.253888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.253927 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.254009 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.254501 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.255223 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.256022 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.256424 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.256531 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.258594 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:54.258602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:54.258616 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:54.258644 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:54.260006 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.260478 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.260484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.260488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.260598 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.260852 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.260854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.260855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.261054 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:54.261194 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:54.261356 (+ 162us) server_negotiation.cc:299] Negotiation successful
0621 07:56:54.261514 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260621 07:56:54.264920 18208 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:54.263905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.11.223.129:46633)
0621 07:56:54.264010 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.264014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.264145 (+ 131us) server_negotiation.cc:407] Connection header received
0621 07:56:54.264291 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.264295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.264379 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.264474 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.264821 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56522: BlockingRecv error: recv got EOF from 127.0.0.1:56522 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260621 07:56:54.265074 12158 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc' of type FILE
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.265933 12158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.266746 12158 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:54.274729 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.267459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.11.223.129:46633)
0621 07:56:54.267564 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.267568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.267662 (+ 94us) server_negotiation.cc:407] Connection header received
0621 07:56:54.267809 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.267821 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.267852 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.267925 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.268448 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.268924 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.269657 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.269884 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.269989 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.272080 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:54.272083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:54.272085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:54.272107 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:54.273265 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.273649 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.273654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.273656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.273731 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.273990 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.273993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.273995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.274138 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:54.274255 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:54.274492 (+ 237us) server_negotiation.cc:299] Negotiation successful
0621 07:56:54.274595 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260621 07:56:54.276952 18208 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:56:54.276077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.11.223.129:46633)
0621 07:56:54.276165 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.276167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.276355 (+ 188us) server_negotiation.cc:407] Connection header received
0621 07:56:54.276463 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.276465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.276496 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.276550 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.276852 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56540: BlockingRecv error: recv got EOF from 127.0.0.1:56540 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260621 07:56:54.276995 12158 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1782028587408853-12158-0/krb5kdc/krb5cc' of type FILE
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.277770 12158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:54.278506 12158 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:56:54.286078 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:54.279159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56556 (local address 127.11.223.129:46633)
0621 07:56:54.279261 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:54.279264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:54.279359 (+ 95us) server_negotiation.cc:407] Connection header received
0621 07:56:54.279493 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:54.279496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:54.279539 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:54.279615 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:54.280070 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.280612 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.281299 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:54.281490 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:54.281585 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:54.283405 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:54.283407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:54.283409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:54.283427 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:54.284601 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.284978 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.284983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.284984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.285055 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:54.285323 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:54.285330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:54.285331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:54.285467 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:54.285602 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:54.285821 (+ 219us) server_negotiation.cc:299] Negotiation successful
0621 07:56:54.285919 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:56:54.287006 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17804
I20260621 07:56:54.292398 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17938
I20260621 07:56:54.297290 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18072
I20260621 07:56:54.302137 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 17710
2026-06-21T07:56:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2035 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[18221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[18221](info): set up 2 sockets
Jun 21 07:56:54 dist-test-slave-0pn3 krb5kdc[18221](info): commencing operation
krb5kdc: starting...
W20260621 07:56:55.300953 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:56:55 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:56:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab.
Entry for principal oryx/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab.
I20260621 07:56:55.412820 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45495
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab
--principal=oryx/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:55.517392 18234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:55.517616 18234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:55.517653 18234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:55.521576 18234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:55.521620 18234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:55.521639 18234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:55.521694 18234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:55.521721 18234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:55.526788 18234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab
--principal=oryx/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:55.527696 18234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:55.528680 18234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:55.536728 18239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:55.536734 18240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:55.536768 18234 server_base.cc:1034] running on GCE node
W20260621 07:56:55.536734 18242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:55.537482 18234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:55.538509 18234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:55.539692 18234 hybrid_clock.cc:648] HybridClock initialized: now 1782028615539664 us; error 48 us; skew 500 ppm
Jun 21 07:56:55 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:55.542297 18234 init.cc:377] Logged in from keytab as oryx/127.11.223.190@KRBTEST.COM (short username oryx)
I20260621 07:56:55.543135 18234 webserver.cc:460] Webserver started at http://127.11.223.190:41653/ using document root <none> and password file <none>
I20260621 07:56:55.543700 18234 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:55.543788 18234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:55.544026 18234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:55.545917 18234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "c46deeb03ea2484491608fb69140b413"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "7593ef78817b699fc6a774ae73b195a2"
server_key_iv: "54f0cee9f39437c238712befd934f781"
server_key_version: "encryptionkey@0"
I20260621 07:56:55.546437 18234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "c46deeb03ea2484491608fb69140b413"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "7593ef78817b699fc6a774ae73b195a2"
server_key_iv: "54f0cee9f39437c238712befd934f781"
server_key_version: "encryptionkey@0"
I20260621 07:56:55.549854 18234 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:56:55.552026 18249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:55.553143 18234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:55.553269 18234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "c46deeb03ea2484491608fb69140b413"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "7593ef78817b699fc6a774ae73b195a2"
server_key_iv: "54f0cee9f39437c238712befd934f781"
server_key_version: "encryptionkey@0"
I20260621 07:56:55.553398 18234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:55.567826 18234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:55.570887 18234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:55.571085 18234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:55.577706 18234 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:45495
I20260621 07:56:55.577955 18301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:45495 every 8 connection(s)
I20260621 07:56:55.578648 18234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:56:55.581239 18302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:55.586316 18302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Bootstrap starting.
I20260621 07:56:55.587953 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18234
I20260621 07:56:55.588097 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:56:55.588433 12158 external_mini_cluster.cc:1428] Setting key 5fb9c552ab5143b5ec8d5e84599bbf88
I20260621 07:56:55.588796 18302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:55.589603 18302 log.cc:824] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:55.591806 18302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: No bootstrap required, opened a new log
I20260621 07:56:55.595961 18302 raft_consensus.cc:348] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } }
I20260621 07:56:55.596153 18302 raft_consensus.cc:374] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:55.596210 18302 raft_consensus.cc:729] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c46deeb03ea2484491608fb69140b413, State: Initialized, Role: FOLLOWER
I20260621 07:56:55.596668 18302 consensus_queue.cc:260] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } }
I20260621 07:56:55.596798 18302 raft_consensus.cc:388] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:56:55.596900 18302 raft_consensus.cc:482] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:56:55.597021 18302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 0 FOLLOWER]: Advancing to term 1
Jun 21 07:56:55 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:56:55.598099 18302 raft_consensus.cc:504] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } }
I20260621 07:56:55.598479 18302 leader_election.cc:302] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c46deeb03ea2484491608fb69140b413; no voters:
I20260621 07:56:55.598766 18302 leader_election.cc:288] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:56:55.599078 18307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:55.599635 18307 raft_consensus.cc:686] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 LEADER]: Becoming Leader. State: Replica: c46deeb03ea2484491608fb69140b413, State: Running, Role: LEADER
I20260621 07:56:55.599921 18302 sys_catalog.cc:564] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:55.600018 18307 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } }
I20260621 07:56:55.601645 18309 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c46deeb03ea2484491608fb69140b413. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } }
I20260621 07:56:55.601809 18308 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } }
I20260621 07:56:55.601816 18309 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:55.601917 18308 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:55.602416 18322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:56:55.604271 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:55.589569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34552 (local address 127.11.223.190:45495)
0621 07:56:55.590043 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:55.590057 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:55.590090 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:56:55.590840 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:55.590861 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:55.591224 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:55.591562 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:55.592265 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:55.593758 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:55.594643 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:55.595353 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:55.595514 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:55.598388 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:55.598394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:55.598405 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:55.598435 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:55.600645 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:55.601167 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:55.601174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:55.601183 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:55.601303 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:55.601621 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:55.601625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:55.601628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:55.602630 (+ 1002us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:55.602836 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:55.603096 (+ 260us) server_negotiation.cc:299] Negotiation successful
0621 07:56:55.603393 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":153,"threads_started":1}
I20260621 07:56:55.605458 18322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:56:55.610601 18322 catalog_manager.cc:1347] Generated new cluster ID: dfbc6cceb4724d06a63caf4c47b2f307
I20260621 07:56:55.610658 18322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:56:55.626210 18322 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:56:55.627411 18322 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:56:55.639410 18322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Generated new TSK 0
I20260621 07:56:55.640192 18322 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab.
Entry for principal oryx/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab.
I20260621 07:56:55.669240 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45495
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:55.778028 18328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:55.778250 18328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:55.778297 18328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:55.782184 18328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:55.782229 18328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:55.782289 18328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:55.787331 18328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:55.788166 18328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:55.789052 18328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:55.798063 18336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:55.798036 18334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:55.798069 18333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:55.798300 18328 server_base.cc:1034] running on GCE node
I20260621 07:56:55.798729 18328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:55.799341 18328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:55.800521 18328 hybrid_clock.cc:648] HybridClock initialized: now 1782028615800508 us; error 24 us; skew 500 ppm
Jun 21 07:56:55 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:55.803316 18328 init.cc:377] Logged in from keytab as oryx/127.11.223.129@KRBTEST.COM (short username oryx)
I20260621 07:56:55.804126 18328 webserver.cc:460] Webserver started at http://127.11.223.129:33425/ using document root <none> and password file <none>
I20260621 07:56:55.804716 18328 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:55.804806 18328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:55.805029 18328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:55.806949 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "668417c1c7a7417fbfd08bc456a59488"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "600418a50eb9467e05a40251ca03c65d"
server_key_iv: "c3c01a70f6fb00d104fa0fdf09b31d8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:55.807499 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "668417c1c7a7417fbfd08bc456a59488"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "600418a50eb9467e05a40251ca03c65d"
server_key_iv: "c3c01a70f6fb00d104fa0fdf09b31d8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:55.811095 18328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260621 07:56:55.813700 18343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:55.814918 18328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:55.815007 18328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "668417c1c7a7417fbfd08bc456a59488"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "600418a50eb9467e05a40251ca03c65d"
server_key_iv: "c3c01a70f6fb00d104fa0fdf09b31d8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:55.815083 18328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:55.826016 18328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:55.828610 18328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:55.828745 18328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:55.829195 18328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:55.829906 18328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:55.829948 18328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:55.829985 18328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:55.830026 18328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:55.838147 18328 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:45175
I20260621 07:56:55.838201 18456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:45175 every 8 connection(s)
I20260621 07:56:55.838984 18328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:55.844547 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18328
I20260621 07:56:55.844677 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:56:55.844980 12158 external_mini_cluster.cc:1428] Setting key 4a2e328f24936c542f8e287be029ec77
Jun 21 07:56:55 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:56:55.853106 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:55.840720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:35845 (local address 127.11.223.190:45495)
0621 07:56:55.840864 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:55.840869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:55.841688 (+ 819us) server_negotiation.cc:407] Connection header received
0621 07:56:55.842938 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:55.842940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:55.842984 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:55.843055 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:55.844190 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:55.844861 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:55.846202 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:55.846487 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:55.846582 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:55.849579 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:55.849582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:55.849584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:55.849606 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:55.851061 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:55.851603 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:55.851607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:55.851609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:55.851689 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:55.852039 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:55.852062 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:55.852064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:55.852233 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:55.852363 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:55.852826 (+ 463us) server_negotiation.cc:299] Negotiation successful
0621 07:56:55.852942 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:56:55.853791 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:55.841086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:35845)
0621 07:56:55.841549 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:55.841587 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:55.842750 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:55.843204 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:55.843210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:55.843528 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:55.844060 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:55.844076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:55.844979 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:55.844982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:55.846067 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:55.846076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:55.846635 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:55.846639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:55.846832 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:55.847398 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:55.847421 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:55.849370 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:55.851200 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:55.851206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:55.851218 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:55.851500 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:55.851791 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:55.851794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:55.851797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:55.851941 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:55.852363 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:55.852371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:55.852589 (+ 218us) client_negotiation.cc:769] Sending connection context
0621 07:56:55.852764 (+ 175us) client_negotiation.cc:240] Negotiation successful
0621 07:56:55.853052 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for oryx/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:56:55.854964 18457 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45495
I20260621 07:56:55.855211 18457 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:55.855664 18457 heartbeater.cc:511] Master 127.11.223.190:45495 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.130@KRBTEST.COM" created.
I20260621 07:56:55.857194 18266 ts_manager.cc:194] Registered new tserver with Master: 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175)
I20260621 07:56:55.858603 18266 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:35845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab.
Entry for principal oryx/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab.
I20260621 07:56:55.870582 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45495
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:55.974928 18462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:55.975139 18462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:55.975183 18462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:55.979020 18462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:55.979068 18462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:55.979120 18462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:55.983966 18462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:55.984973 18462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:55.985936 18462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:55.995172 18467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:55.995167 18470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:55.995270 18468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:55.995436 18462 server_base.cc:1034] running on GCE node
I20260621 07:56:55.995862 18462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:55.996542 18462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:55.997721 18462 hybrid_clock.cc:648] HybridClock initialized: now 1782028615997711 us; error 28 us; skew 500 ppm
Jun 21 07:56:55 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:56.000394 18462 init.cc:377] Logged in from keytab as oryx/127.11.223.130@KRBTEST.COM (short username oryx)
I20260621 07:56:56.001318 18462 webserver.cc:460] Webserver started at http://127.11.223.130:42487/ using document root <none> and password file <none>
I20260621 07:56:56.001861 18462 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.001948 18462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.002182 18462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:56.004674 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "88efbf17dec149638cba8a3b8fc78145"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "0a30ff7aefe2c818ef02de0bebbdc9a9"
server_key_iv: "9d740a71ac05f9daf084e0006984e434"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.005218 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "88efbf17dec149638cba8a3b8fc78145"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "0a30ff7aefe2c818ef02de0bebbdc9a9"
server_key_iv: "9d740a71ac05f9daf084e0006984e434"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.008842 18462 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:56.011288 18477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.012607 18462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:56.012742 18462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "88efbf17dec149638cba8a3b8fc78145"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "0a30ff7aefe2c818ef02de0bebbdc9a9"
server_key_iv: "9d740a71ac05f9daf084e0006984e434"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.012868 18462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.039217 18462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:56.041956 18462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:56.042171 18462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:56.042744 18462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:56.043536 18462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:56.043606 18462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.043675 18462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:56.043728 18462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.052345 18462 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:42267
I20260621 07:56:56.052393 18590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:42267 every 8 connection(s)
I20260621 07:56:56.053208 18462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:56.056224 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18462
I20260621 07:56:56.056375 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:56:56.056712 12158 external_mini_cluster.cc:1428] Setting key 201ad550c5c8e232c528f421c197e383
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.131@KRBTEST.COM" created.
I20260621 07:56:56.068658 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.054948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:39409 (local address 127.11.223.190:45495)
0621 07:56:56.055077 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.055080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.055916 (+ 836us) server_negotiation.cc:407] Connection header received
0621 07:56:56.058694 (+ 2778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.058697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.058735 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.058791 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.060293 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.061159 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.062297 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.062668 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.062793 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.065118 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.065122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.065125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.065152 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.066713 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.067164 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.067174 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.067176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.067251 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.067577 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.067582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.067583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.067768 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.067876 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.068363 (+ 487us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.068489 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:56:56.069375 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.055302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:39409)
0621 07:56:56.055755 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.055815 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.058482 (+ 2667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.058956 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.058968 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.059481 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.060155 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.060170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.061273 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.061275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.062158 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.062169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.062801 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.062804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.062935 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.063497 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.063512 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.064957 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.066807 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.066812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.066822 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.067049 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.067353 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.067360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.067362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.067482 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.067873 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.067878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.068097 (+ 219us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.068284 (+ 187us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.068604 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
I20260621 07:56:56.070814 18591 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45495
I20260621 07:56:56.071043 18591 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:56.071476 18591 heartbeater.cc:511] Master 127.11.223.190:45495 requested a full tablet report, sending...
I20260621 07:56:56.072589 18266 ts_manager.cc:194] Registered new tserver with Master: 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267)
I20260621 07:56:56.073144 18266 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:39409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab.
Entry for principal oryx/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab.
I20260621 07:56:56.082876 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45495
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:56.192109 18596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:56.192374 18596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:56.192459 18596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:56.196471 18596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:56.196540 18596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:56.196625 18596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:56.202062 18596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:56.202965 18596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:56.203890 18596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:56.212841 18604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.212838 18601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.212848 18602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.213246 18596 server_base.cc:1034] running on GCE node
I20260621 07:56:56.213589 18596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:56.214099 18596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:56.215294 18596 hybrid_clock.cc:648] HybridClock initialized: now 1782028616215284 us; error 28 us; skew 500 ppm
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:56.218072 18596 init.cc:377] Logged in from keytab as oryx/127.11.223.131@KRBTEST.COM (short username oryx)
I20260621 07:56:56.218793 18596 webserver.cc:460] Webserver started at http://127.11.223.131:36585/ using document root <none> and password file <none>
I20260621 07:56:56.219345 18596 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.219408 18596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.219566 18596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:56.221454 18596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "9b442adc28634d16bf30af967fe1c4ea"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "0532838531d214cb0220951413ce8df1"
server_key_iv: "df8545e7f47354f37e8b767afbd739cf"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.221923 18596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "9b442adc28634d16bf30af967fe1c4ea"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "0532838531d214cb0220951413ce8df1"
server_key_iv: "df8545e7f47354f37e8b767afbd739cf"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.225330 18596 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:56.227523 18611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.228751 18596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:56:56.228843 18596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "9b442adc28634d16bf30af967fe1c4ea"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "0532838531d214cb0220951413ce8df1"
server_key_iv: "df8545e7f47354f37e8b767afbd739cf"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.228921 18596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.246901 18596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:56.249917 18596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:56.250056 18596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:56.250527 18596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:56.251210 18596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:56.251255 18596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.251302 18596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:56.251322 18596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.259991 18596 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:45215
I20260621 07:56:56.260069 18724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:45215 every 8 connection(s)
I20260621 07:56:56.260895 18596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:56.268714 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18596
I20260621 07:56:56.268807 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:56:56.269057 12158 external_mini_cluster.cc:1428] Setting key 2f18a9af1bf83ee1280abf3e39e4a7db
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:56:56.273443 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.262556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:60271 (local address 127.11.223.190:45495)
0621 07:56:56.262694 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.262699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.263480 (+ 781us) server_negotiation.cc:407] Connection header received
0621 07:56:56.264724 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.264727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.264776 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.264849 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.265870 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.266390 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.267420 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.267623 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.267704 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.270252 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.270257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.270268 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.270295 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.271702 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.272132 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.272134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.272136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.272204 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.272545 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.272547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.272548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.272716 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.272794 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.273237 (+ 443us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.273332 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:56:56.274103 18727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.262957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:60271)
0621 07:56:56.263343 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.263381 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.264550 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.264999 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.265005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.265315 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.265751 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.265762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.266487 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.266489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.267325 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.267333 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.267742 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.267745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.267875 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.268360 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.268374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.270064 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.271799 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.271803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.271813 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.272050 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.272278 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.272280 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.272281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.272466 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.272797 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.272802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.272993 (+ 191us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.273148 (+ 155us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.273361 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":93,"threads_started":1}
I20260621 07:56:56.275079 18725 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45495
I20260621 07:56:56.275267 18725 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:56.275637 18725 heartbeater.cc:511] Master 127.11.223.190:45495 requested a full tablet report, sending...
I20260621 07:56:56.276571 18266 ts_manager.cc:194] Registered new tserver with Master: 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215)
I20260621 07:56:56.277060 18266 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:60271
I20260621 07:56:56.282495 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.223.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.189.keytab.
Entry for principal oryx/127.11.223.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.189.keytab.
I20260621 07:56:56.307283 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:43445
--webserver_interface=127.11.223.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.189.keytab
--principal=oryx/127.11.223.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45495,127.11.223.189:43445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:56:56.414568 18730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:56.414839 18730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:56.414945 18730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:56.419121 18730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:56.419190 18730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:56.419226 18730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:56.419284 18730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:56.419322 18730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:56.424485 18730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45495,127.11.223.189:43445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.189.keytab
--principal=oryx/127.11.223.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:43445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.223.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
W20260621 07:56:56.424700 18730 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.223.190:45495,127.11.223.189:43445'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260621 07:56:56.425495 18730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:56.426450 18730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:56.434530 18735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.434538 18736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.434587 18730 server_base.cc:1034] running on GCE node
W20260621 07:56:56.434548 18738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.435209 18730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:56.436168 18730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:56.437319 18730 hybrid_clock.cc:648] HybridClock initialized: now 1782028616437296 us; error 39 us; skew 500 ppm
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:56.440004 18730 init.cc:377] Logged in from keytab as oryx/127.11.223.189@KRBTEST.COM (short username oryx)
I20260621 07:56:56.440925 18730 webserver.cc:460] Webserver started at http://127.11.223.189:44897/ using document root <none> and password file <none>
I20260621 07:56:56.441478 18730 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.441566 18730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.441799 18730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:56.443778 18730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/instance:
uuid: "3f33f8156bbc41968bef95bf0765d392"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "b0226d0c6d431eaab27183c0a3311cbc"
server_key_iv: "7620f5c6c2b2ad624ad4c2b350595ae2"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.444350 18730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal/instance:
uuid: "3f33f8156bbc41968bef95bf0765d392"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "b0226d0c6d431eaab27183c0a3311cbc"
server_key_iv: "7620f5c6c2b2ad624ad4c2b350595ae2"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.447844 18730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:56:56.450145 18745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.451221 18730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:56.451357 18730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
uuid: "3f33f8156bbc41968bef95bf0765d392"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "b0226d0c6d431eaab27183c0a3311cbc"
server_key_iv: "7620f5c6c2b2ad624ad4c2b350595ae2"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.451460 18730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.467790 18730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:56.470804 18730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:56.470986 18730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:56.477658 18730 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.189:43445
I20260621 07:56:56.477934 18797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.189:43445 every 8 connection(s)
I20260621 07:56:56.478555 18730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
I20260621 07:56:56.481084 18798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:56.482445 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18730
I20260621 07:56:56.485146 18798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.189@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:56:56.496251 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.487000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.11.223.190:45495)
0621 07:56:56.487123 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.487126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.487446 (+ 320us) server_negotiation.cc:407] Connection header received
0621 07:56:56.487605 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.487608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.487658 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.487745 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.488350 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.488857 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.489861 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.490234 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.490362 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.492646 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.492651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.492655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.492687 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.494296 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.494865 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.494869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.494871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.494957 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.495217 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.495219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.495220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.495505 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.495613 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.496054 (+ 441us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.496140 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:56:56.496994 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.483909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.11.223.189:43445)
0621 07:56:56.484279 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.484294 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.484368 (+ 74us) server_negotiation.cc:407] Connection header received
0621 07:56:56.484966 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.484981 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.485280 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.485605 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.486427 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.487778 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.488684 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.489149 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.489321 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.492165 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.492171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.492179 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.492199 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.493894 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.494399 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.494404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.494406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.494588 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.494891 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.494893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.494894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.495339 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.495484 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.495792 (+ 308us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.496051 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"spinlock_wait_cycles":18432,"thread_start_us":117,"threads_started":1}
I20260621 07:56:56.496994 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.486933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:34568)
0621 07:56:56.487323 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.487362 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.487502 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.487791 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.487797 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.488102 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.488207 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.488212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.488994 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.488998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.489740 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.489752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.490393 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.490395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.490480 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.491006 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.491026 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.492511 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.494392 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.494399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.494407 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.494770 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.495042 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.495045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.495047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.495130 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.495611 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.495635 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.495787 (+ 152us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.495963 (+ 176us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.496101 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"spinlock_wait_cycles":7168,"thread_start_us":115,"threads_started":1}
I20260621 07:56:56.498276 18798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.189@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
I20260621 07:56:56.506022 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.498732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38454)
0621 07:56:56.498968 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.498981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.499046 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.499278 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.499280 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.499398 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.499480 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.499485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.500632 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.500635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.501229 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.501238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.501680 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.501683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.501768 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.502092 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.502102 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.503207 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.504659 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.504664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.504666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.504846 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.505115 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.505120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.505122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.505199 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.505544 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.505546 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.505621 (+ 75us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.505733 (+ 112us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.505855 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260621 07:56:56.506031 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.498881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.11.223.189:43445)
0621 07:56:56.499131 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.499133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.499140 (+ 7us) server_negotiation.cc:407] Connection header received
0621 07:56:56.499158 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.499160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.499183 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.499233 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.499584 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.500509 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.501358 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.501583 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.501685 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.503337 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.503340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.503343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.503363 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.504548 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.504943 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.504952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.504954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.505026 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.505294 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.505300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.505302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.505450 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.505553 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.505782 (+ 229us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.505907 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":211}
I20260621 07:56:56.508579 18798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Bootstrap starting.
I20260621 07:56:56.510584 18798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:56.511216 18798 log.cc:824] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:56.512902 18798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: No bootstrap required, opened a new log
I20260621 07:56:56.515637 18798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } }
I20260621 07:56:56.515810 18798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:56.515841 18798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f33f8156bbc41968bef95bf0765d392, State: Initialized, Role: FOLLOWER
I20260621 07:56:56.516168 18798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } }
I20260621 07:56:56.516601 18804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } } }
I20260621 07:56:56.516724 18804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:56:56.517298 18798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: configured and running, proceeding with master startup.
W20260621 07:56:56.521564 18815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:56:56.521648 18815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:56:56.523329 18730 master_runner.cc:428] Detected that this master 3f33f8156bbc41968bef95bf0765d392 is joining an existing cluster
I20260621 07:56:56.523396 18730 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.223.189:43445
I20260621 07:56:56.533377 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.525568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.11.223.190:45495)
0621 07:56:56.525690 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.525694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.525930 (+ 236us) server_negotiation.cc:407] Connection header received
0621 07:56:56.526095 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.526098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.526147 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.526240 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.526744 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.527322 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.528142 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.528451 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.528563 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.529621 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.529625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.529638 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.529662 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.531399 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.531963 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.531966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.531968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.532055 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.532399 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.532405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.532409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.532726 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.532845 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.533099 (+ 254us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.533217 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 07:56:56.533390 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.525478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:34578)
0621 07:56:56.525859 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.525873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.525970 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.526255 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.526258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.526481 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.526596 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.526602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.527429 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.527435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.528021 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.528029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.528572 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.528575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.528668 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.529163 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.529175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.529485 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.531520 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.531523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.531525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.531848 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.532165 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.532168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.532170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.532255 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.532834 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.532837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.532931 (+ 94us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.533037 (+ 106us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.533217 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":101,"threads_started":1}
I20260621 07:56:56.533885 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.525962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.11.223.189:43445)
0621 07:56:56.526065 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.526069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.526176 (+ 107us) server_negotiation.cc:407] Connection header received
0621 07:56:56.526310 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.526313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.526346 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.526391 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.527043 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.527771 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.528965 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.529276 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.529352 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.530541 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.530547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.530551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.530579 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.532187 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.532701 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.532704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.532706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.532798 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.533087 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.533089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.533091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.533261 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.533328 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.533632 (+ 304us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.533731 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 07:56:56.534334 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.525867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38462)
0621 07:56:56.526097 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.526108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.526223 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.526526 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.526529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.526787 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.526920 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.526928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.527909 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.527912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.528847 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.528857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.529474 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.529477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.529567 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.530018 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.530033 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.530407 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.532303 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.532307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.532333 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.532564 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.532901 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.532904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.532906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.532996 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.533388 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.533392 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.533466 (+ 74us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.534013 (+ 547us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.534161 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"spinlock_wait_cycles":2944,"thread_start_us":83,"threads_started":1}
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.190@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
I20260621 07:56:56.546391 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.537748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.11.223.189:43445)
0621 07:56:56.537866 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.537873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.538127 (+ 254us) server_negotiation.cc:407] Connection header received
0621 07:56:56.538293 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.538296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.538349 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.538428 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.538970 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.539717 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.540657 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.540987 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.541085 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.543239 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.543243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.543246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.543270 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.544750 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.545164 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.545167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.545169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.545250 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.545578 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.545580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.545581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.545733 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.545813 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.546199 (+ 386us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.546292 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 07:56:56.546489 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.537708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38472)
0621 07:56:56.538007 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.538051 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.538173 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.538461 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.538465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.538774 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.538862 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.538867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.539860 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.539863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.540545 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.540555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.541091 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.541093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.541166 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.541637 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.541649 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.543104 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.544855 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.544862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.544867 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.545073 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.545383 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.545386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.545388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.545487 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.545836 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.545842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.545960 (+ 118us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.546138 (+ 178us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.546316 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260621 07:56:56.547762 18266 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.223.189:43445: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3f33f8156bbc41968bef95bf0765d392"
member_type: NON_VOTER
last_known_addr {
host: "127.11.223.189"
port: 43445
}
attrs {
promote: true
}
}
dest_uuid: "c46deeb03ea2484491608fb69140b413"
cas_config_opid_index: -1
I20260621 07:56:56.548452 18266 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } }
I20260621 07:56:56.550089 18824 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 3f33f8156bbc41968bef95bf0765d392 (127.11.223.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } }
I20260621 07:56:56.550974 18772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:56.552070 18824 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.223.189:43445
I20260621 07:56:56.552402 18772 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Refusing update from remote peer c46deeb03ea2484491608fb69140b413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260621 07:56:56.552487 18730 master.cc:537] Master@127.11.223.189:43445 shutting down...
I20260621 07:56:56.552868 18824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } }
I20260621 07:56:56.552968 18824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:56.553130 18824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 07:56:56.555053 18252 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260621 07:56:56.556880 18730 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:56.557176 18730 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:56.557317 18730 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: stopping tablet replica
I20260621 07:56:56.561350 18730 master.cc:552] Master@127.11.223.189:43445 shutdown complete.
I20260621 07:56:56.561411 18730 master_runner.cc:305] Clearing existing system tablet
WARNING: no policy specified for oryx/127.11.223.188@KRBTEST.COM; defaulting to no policy
I20260621 07:56:56.561664 18730 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.561731 18730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.562847 18730 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.188@KRBTEST.COM" created.
I20260621 07:56:56.564049 18831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.564244 18730 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.564628 18730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:56.564719 18730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
uuid: "3f33f8156bbc41968bef95bf0765d392"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "b0226d0c6d431eaab27183c0a3311cbc"
server_key_iv: "7620f5c6c2b2ad624ad4c2b350595ae2"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.565743 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.564934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38488)
0621 07:56:56.565603 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.565669 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588}
W20260621 07:56:56.565946 18252 consensus_peers.cc:487] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 -> Peer 3f33f8156bbc41968bef95bf0765d392 (127.11.223.189:43445): Couldn't send request to peer 3f33f8156bbc41968bef95bf0765d392. Status: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260621 07:56:56.566946 18730 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:56.568823 18730 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260621 07:56:56.568920 18730 log.cc:1196] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260621 07:56:56.569154 18730 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Deleting consensus metadata
I20260621 07:56:56.569335 18730 master_runner.cc:315] Copying system tablet from 127.11.223.190:45495
I20260621 07:56:56.570375 18730 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: tablet copy: Beginning tablet copy session from remote peer at address 127.11.223.190:45495
I20260621 07:56:56.578346 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.570875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.11.223.190:45495)
0621 07:56:56.570985 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.570990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.571200 (+ 210us) server_negotiation.cc:407] Connection header received
0621 07:56:56.571365 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.571368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.571412 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.571497 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.571905 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.572520 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.573441 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.573764 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.573871 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.575038 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.575041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.575044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.575067 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.576498 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.576921 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.576926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.576927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.577079 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.577354 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.577357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.577358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.577618 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.577708 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.578065 (+ 357us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.578174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.188.keytab.
Entry for principal oryx/127.11.223.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.188.keytab.
I20260621 07:56:56.579069 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.570818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:34592)
0621 07:56:56.571117 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.571128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.571253 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.571510 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.571513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.571707 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.571797 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.571802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.572625 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.572631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.573326 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.573335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.573872 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.573876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.573973 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.574522 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.574534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.574906 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.576613 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.576616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.576618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.576824 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.577168 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.577173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.577175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.577259 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.577751 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.577754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.577884 (+ 130us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.578758 (+ 874us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.578891 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":91,"threads_started":1}
I20260621 07:56:56.579324 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:42685
--webserver_interface=127.11.223.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.188.keytab
--principal=oryx/127.11.223.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:56:56.579697 18286 tablet_copy_service.cc:140] P c46deeb03ea2484491608fb69140b413: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3f33f8156bbc41968bef95bf0765d392 ({username='oryx', principal='oryx/127.11.223.189@KRBTEST.COM'} at 127.0.0.1:34592)
I20260621 07:56:56.579839 18286 tablet_copy_service.cc:161] P c46deeb03ea2484491608fb69140b413: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3f33f8156bbc41968bef95bf0765d392 at {username='oryx', principal='oryx/127.11.223.189@KRBTEST.COM'} at 127.0.0.1:34592: session id = 3f33f8156bbc41968bef95bf0765d392-00000000000000000000000000000000
I20260621 07:56:56.581396 18286 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Tablet Copy: opened 0 blocks and 1 log segments
I20260621 07:56:56.582837 18730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260621 07:56:56.584847 18730 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: tablet copy: Starting download of 0 data blocks...
I20260621 07:56:56.584995 18730 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: tablet copy: Starting download of 1 WAL segments...
I20260621 07:56:56.589727 18730 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260621 07:56:56.591331 18286 tablet_copy_service.cc:342] P c46deeb03ea2484491608fb69140b413: Request end of tablet copy session 3f33f8156bbc41968bef95bf0765d392-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.223.189@KRBTEST.COM'} at 127.0.0.1:34592
I20260621 07:56:56.591421 18286 tablet_copy_service.cc:433] P c46deeb03ea2484491608fb69140b413: ending tablet copy session 3f33f8156bbc41968bef95bf0765d392-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3f33f8156bbc41968bef95bf0765d392
I20260621 07:56:56.598757 18730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:56.600318 18840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.600301 18841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.600804 18843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.600968 18730 server_base.cc:1034] running on GCE node
I20260621 07:56:56.601195 18730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:56.601492 18730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:56.602677 18730 hybrid_clock.cc:648] HybridClock initialized: now 1782028616602665 us; error 28 us; skew 500 ppm
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:56.604115 18730 init.cc:377] Logged in from keytab as oryx/127.11.223.189@KRBTEST.COM (short username oryx)
I20260621 07:56:56.604573 18730 webserver.cc:460] Webserver started at http://127.11.223.189:36279/ using document root <none> and password file <none>
I20260621 07:56:56.604765 18730 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.604823 18730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.605866 18730 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260621 07:56:56.606707 18850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.606922 18730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260621 07:56:56.607004 18730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
uuid: "3f33f8156bbc41968bef95bf0765d392"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "b0226d0c6d431eaab27183c0a3311cbc"
server_key_iv: "7620f5c6c2b2ad624ad4c2b350595ae2"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.607075 18730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.613256 18730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:56.615132 18730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:56.619758 18730 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.189:43445
I20260621 07:56:56.620235 18730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
I20260621 07:56:56.621572 18903 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:56:56.622380 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Bootstrap starting.
I20260621 07:56:56.631865 18902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.189:43445 every 8 connection(s)
I20260621 07:56:56.632200 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:56:56.632556 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Bootstrap complete.
I20260621 07:56:56.633086 18903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } }
I20260621 07:56:56.633211 18903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3f33f8156bbc41968bef95bf0765d392, State: Initialized, Role: LEARNER
I20260621 07:56:56.633340 18903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } }
I20260621 07:56:56.633666 18906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } }
I20260621 07:56:56.633816 18906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: LEARNER
I20260621 07:56:56.633838 18903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:56.636932 18917 catalog_manager.cc:1259] Loaded cluster ID: dfbc6cceb4724d06a63caf4c47b2f307
I20260621 07:56:56.637003 18917 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: loading cluster ID for follower catalog manager: success
I20260621 07:56:56.638424 18917 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: acquiring CA information for follower catalog manager: success
I20260621 07:56:56.639436 18917 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260621 07:56:56.730854 18839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:56.731154 18839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:56.731212 18839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:56.736653 18839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:56:56.736708 18839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:56.736727 18839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:56.736744 18839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:56:56.736759 18839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:56:56.741814 18839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.188.keytab
--principal=oryx/127.11.223.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:42685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.223.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:56.742671 18839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:56.743558 18839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:56.751860 18925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.751881 18923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.751973 18922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.752094 18839 server_base.cc:1034] running on GCE node
I20260621 07:56:56.752687 18839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:56.753664 18839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:56.754859 18839 hybrid_clock.cc:648] HybridClock initialized: now 1782028616754829 us; error 48 us; skew 500 ppm
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:56.757709 18839 init.cc:377] Logged in from keytab as oryx/127.11.223.188@KRBTEST.COM (short username oryx)
I20260621 07:56:56.758440 18839 webserver.cc:460] Webserver started at http://127.11.223.188:34587/ using document root <none> and password file <none>
I20260621 07:56:56.758903 18839 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.758956 18839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.759099 18839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:56:56.761036 18839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/instance:
uuid: "5a30916872334f0aa08d515def96dc57"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "1a2b5650c37c8c119cd44a79a8292429"
server_key_iv: "62f19ce9f65ce35ec8171b10195659c5"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.761519 18839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal/instance:
uuid: "5a30916872334f0aa08d515def96dc57"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "1a2b5650c37c8c119cd44a79a8292429"
server_key_iv: "62f19ce9f65ce35ec8171b10195659c5"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.766799 18839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260621 07:56:56.769572 18932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.770875 18839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:56:56.770996 18839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
uuid: "5a30916872334f0aa08d515def96dc57"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "1a2b5650c37c8c119cd44a79a8292429"
server_key_iv: "62f19ce9f65ce35ec8171b10195659c5"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.771096 18839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.794420 18839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:56.797293 18839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:56.797474 18839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:56.803951 18839 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.188:42685
I20260621 07:56:56.804288 18984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.188:42685 every 8 connection(s)
I20260621 07:56:56.805133 18839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
I20260621 07:56:56.806243 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 18839
I20260621 07:56:56.807996 18985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:56.811462 18985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.223.188@KRBTEST.COM
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.188@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:56:56.821036 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.807619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.11.223.188:42685)
0621 07:56:56.808008 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.808021 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.808052 (+ 31us) server_negotiation.cc:407] Connection header received
0621 07:56:56.808848 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.808863 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.809150 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.809477 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.810385 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.811322 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.812264 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.812766 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.812930 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.815591 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.815597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.815610 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.815640 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.817822 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.818320 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.818324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.818328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.818432 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.818795 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.818798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.818800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.819227 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.819446 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.819781 (+ 335us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.820049 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":128,"threads_started":1}
I20260621 07:56:56.822549 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.813187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.11.223.190:45495)
0621 07:56:56.813342 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.813349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.813765 (+ 416us) server_negotiation.cc:407] Connection header received
0621 07:56:56.813881 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.813885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.813943 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.814052 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.814692 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.815268 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.816213 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.816588 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.816705 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.819120 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.819124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.819126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.819152 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.820806 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.821295 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.821298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.821302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.821382 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.821660 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.821664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.821666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.821833 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.821937 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.822285 (+ 348us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.822388 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 07:56:56.822561 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.813064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:34608)
0621 07:56:56.813624 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.813669 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.813791 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.814090 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.814094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.814399 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.814562 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.814572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.815408 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.815411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.815998 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.816095 (+ 97us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.816717 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.816721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.816872 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.817311 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.817325 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.818986 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.820908 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.820915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.820918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.821165 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.821482 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.821485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.821487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.821571 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.821945 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.821954 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.822067 (+ 113us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.822210 (+ 143us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.822375 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":120,"threads_started":1}
I20260621 07:56:56.823701 18985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.188@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
I20260621 07:56:56.830948 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.824164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38492)
0621 07:56:56.824280 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.824292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.824403 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.824738 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.824740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.824862 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.824938 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.824942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.825805 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.825807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.826310 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.826316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.826745 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.826755 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.826819 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.827168 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.827179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.828226 (+ 1047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.829793 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.829796 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.829797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.829972 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.830197 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.830199 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.830201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.830270 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.830576 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.830578 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.830641 (+ 63us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.830716 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.830816 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260621 07:56:56.831055 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.824290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.11.223.189:43445)
0621 07:56:56.824555 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.824558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.824570 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:56:56.824606 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.824607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.824643 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.824716 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.825053 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.825718 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.826429 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.826650 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.826729 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.828397 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.828400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.828402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.828423 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.829699 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.830053 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.830055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.830056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.830123 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.830342 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.830344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.830345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.830501 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.830573 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.830775 (+ 202us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.830901 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":119,"threads_started":1}
I20260621 07:56:56.831655 18985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.188@KRBTEST.COM for oryx/127.11.223.188@KRBTEST.COM
I20260621 07:56:56.839419 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.832080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:48964)
0621 07:56:56.832187 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.832202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.832368 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.832561 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.832563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.832675 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.832735 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.832739 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.833807 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.833809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.834392 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.834407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.834794 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.834797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.834867 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.835277 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.835295 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.836388 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.837963 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.837966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.837967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.838230 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.838506 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.838508 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.838510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.838591 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.838933 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.838936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.839018 (+ 82us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.839130 (+ 112us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.839248 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260621 07:56:56.839421 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.832182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.11.223.188:42685)
0621 07:56:56.832286 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.832289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.832300 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:56.832457 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.832459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.832490 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.832546 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.832817 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.833706 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.834492 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.834691 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.834795 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.836491 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.836494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.836495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.836515 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.837870 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.838323 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.838325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.838326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.838408 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.838671 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.838673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.838674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.838839 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.838908 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.839183 (+ 275us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.839306 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":37}
I20260621 07:56:56.841907 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Bootstrap starting.
I20260621 07:56:56.844085 18985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:56.844842 18985 log.cc:824] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:56.846526 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: No bootstrap required, opened a new log
I20260621 07:56:56.849505 18985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } }
I20260621 07:56:56.849725 18985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:56.849808 18985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a30916872334f0aa08d515def96dc57, State: Initialized, Role: FOLLOWER
I20260621 07:56:56.850200 18985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } }
I20260621 07:56:56.850618 18991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } } }
I20260621 07:56:56.850721 18991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:56:56.851291 18985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: configured and running, proceeding with master startup.
W20260621 07:56:56.855974 19002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:56:56.856073 19002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:56:56.859369 18839 master_runner.cc:428] Detected that this master 5a30916872334f0aa08d515def96dc57 is joining an existing cluster
I20260621 07:56:56.859429 18839 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.223.188:42685
I20260621 07:56:56.860728 18457 heartbeater.cc:503] Master 127.11.223.190:45495 was elected leader, sending a full tablet report...
I20260621 07:56:56.870976 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.862331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:48974)
0621 07:56:56.862745 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.862758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.862837 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.863420 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.863426 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.863660 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.863773 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.863780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.864773 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.864776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.866481 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.866494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.867104 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.867106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.867175 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.867622 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.867640 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.867952 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.869726 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.869729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.869730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.869910 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.870181 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.870183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.870184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.870253 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.870613 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.870615 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.870681 (+ 66us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.870758 (+ 77us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.870846 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":531,"thread_start_us":77,"threads_started":1}
I20260621 07:56:56.871105 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.862720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.11.223.188:42685)
0621 07:56:56.863175 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.863180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.863194 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:56:56.863225 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.863228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.863264 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.863338 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.863884 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.864674 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.866615 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.866905 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.866984 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.868081 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.868085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.868088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.868111 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.869603 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.869999 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.870003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.870006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.870092 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.870335 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.870338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.870340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.870524 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.870618 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.870833 (+ 215us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.870933 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":379}
I20260621 07:56:56.873714 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18328
I20260621 07:56:56.879868 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:45175
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:45495
--webserver_port=33425
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:56:56.888409 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.861642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.11.223.190:45495)
0621 07:56:56.861795 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.861800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.862025 (+ 225us) server_negotiation.cc:407] Connection header received
0621 07:56:56.862193 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.862196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.862241 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.862291 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.862923 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.863728 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.865069 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.865412 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.865487 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.884844 (+ 19357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.884849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.884851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.884878 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.886550 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.887047 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.887050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.887052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.887161 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.887484 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.887487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.887489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.887691 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.887771 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.888099 (+ 328us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.888219 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:56:56.889106 19008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.861590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:34622)
0621 07:56:56.861946 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.861966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.862086 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.862438 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.862441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.862651 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.862782 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.862790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.863881 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.863884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.864936 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.864947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.865648 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.865651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.866410 (+ 759us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.867053 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.867069 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.884672 (+ 17603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.886671 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.886674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.886676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.886923 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.887277 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.887279 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.887281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.887379 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.887841 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.887844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.887928 (+ 84us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.888780 (+ 852us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.888917 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":29,"spinlock_wait_cycles":2944,"thread_start_us":106,"threads_started":1}
I20260621 07:56:56.890295 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.862623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.11.223.189:43445)
0621 07:56:56.862832 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.862836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.862855 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:56:56.862899 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.862901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.862947 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.863015 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.863509 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.864341 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.865488 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.865883 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.865956 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.886099 (+ 20143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.886106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.886110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.886142 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.888101 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.888709 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.888715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.888719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.888810 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.889163 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.889168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.889170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.889415 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.889498 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.889937 (+ 439us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.890081 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260621 07:56:56.891036 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.861987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38502)
0621 07:56:56.862284 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.862299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.862381 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.863025 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.863028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.863244 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.863379 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.863388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.864465 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.864468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.865363 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.865371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.866072 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.866075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.866514 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.884850 (+ 18336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.884870 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.885917 (+ 1047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.888265 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.888268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.888269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.888559 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.888939 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.888942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.888944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.889043 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.889581 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.889584 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.889721 (+ 137us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.890709 (+ 988us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.890869 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":338,"thread_start_us":94,"threads_started":1}
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.190@KRBTEST.COM for oryx/127.11.223.188@KRBTEST.COM
I20260621 07:56:56.901726 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.892785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:48986)
0621 07:56:56.892933 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.892946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.893035 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.893337 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.893341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.893514 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.893607 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.893614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.894563 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.894566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.895448 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.895458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.896008 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.896011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.896113 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.896579 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.896593 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.898172 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.900094 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.900098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.900100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.900430 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.900765 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.900769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.900773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.900867 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.901245 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.901248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.901335 (+ 87us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.901439 (+ 104us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.901561 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260621 07:56:56.901782 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.892850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48986 (local address 127.11.223.188:42685)
0621 07:56:56.892938 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.892942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.893109 (+ 167us) server_negotiation.cc:407] Connection header received
0621 07:56:56.893178 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.893181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.893226 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.893296 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.893721 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.894455 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.895563 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.895900 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.895993 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.898314 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.898325 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.898327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.898350 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.899975 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.900518 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.900524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.900526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.900608 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.900961 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.900964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.900966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.901147 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.901242 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.901519 (+ 277us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.901624 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260621 07:56:56.903038 18266 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.223.188:42685: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5a30916872334f0aa08d515def96dc57"
member_type: NON_VOTER
last_known_addr {
host: "127.11.223.188"
port: 42685
}
attrs {
promote: true
}
}
dest_uuid: "c46deeb03ea2484491608fb69140b413"
cas_config_opid_index: 5
I20260621 07:56:56.903615 18266 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } }
I20260621 07:56:56.904593 18825 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5a30916872334f0aa08d515def96dc57 (127.11.223.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.905916 18825 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.223.188:42685
I20260621 07:56:56.906052 18824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.906155 18824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:56.906471 18839 master.cc:537] Master@127.11.223.188:42685 shutting down...
I20260621 07:56:56.906462 18959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:56.908025 18959 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Refusing update from remote peer c46deeb03ea2484491608fb69140b413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260621 07:56:56.909032 18824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260621 07:56:56.910859 18252 consensus_peers.cc:487] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 -> Peer 5a30916872334f0aa08d515def96dc57 (127.11.223.188:42685): Couldn't send request to peer 5a30916872334f0aa08d515def96dc57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260621 07:56:56.911091 18839 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:56.911340 18839 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:56.911437 18839 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: stopping tablet replica
I20260621 07:56:56.913167 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.904945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38510)
0621 07:56:56.905041 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.905055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.905123 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.909127 (+ 4004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.909130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.909147 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:56.909218 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.909224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.910219 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.910223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.911706 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.911715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.912787 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.912790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.912884 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.912924 (+ 40us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.912997 (+ 73us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.913059 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260621 07:56:56.913558 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.908503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.11.223.189:43445)
0621 07:56:56.908775 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.908782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.908798 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:56:56.908841 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.908846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.908914 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.909007 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:56.909364 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.910072 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.911833 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.912482 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.912589 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.912712 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.913379 (+ 667us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.913452 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260621 07:56:56.914286 18876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 LEARNER]: Refusing update from remote peer c46deeb03ea2484491608fb69140b413: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260621 07:56:56.915020 18824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260621 07:56:56.917071 18876 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5a30916872334f0aa08d515def96dc57 (127.11.223.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.919258 18906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c46deeb03ea2484491608fb69140b413. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.919363 18906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: LEARNER
I20260621 07:56:56.919468 18906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: NON_VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: true } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.919533 18906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: LEARNER
I20260621 07:56:56.920215 18825 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: attempting to promote NON_VOTER 3f33f8156bbc41968bef95bf0765d392 to VOTER
I20260621 07:56:56.920627 18825 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } }
I20260621 07:56:56.921623 18876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 LEARNER]: Refusing update from remote peer c46deeb03ea2484491608fb69140b413: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260621 07:56:56.921679 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.921464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:49000)
0621 07:56:56.921569 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.921621 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:42685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260621 07:56:56.921993 18824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260621 07:56:56.922612 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.922199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:49006)
0621 07:56:56.922508 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.922555 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:42685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
W20260621 07:56:56.922749 18252 consensus_peers.cc:487] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 -> Peer 5a30916872334f0aa08d515def96dc57 (127.11.223.188:42685): Couldn't send request to peer 5a30916872334f0aa08d515def96dc57. Status: Network error: Client connection negotiation failed: client connection to 127.11.223.188:42685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260621 07:56:56.923650 18825 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3f33f8156bbc41968bef95bf0765d392 (127.11.223.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.924069 18876 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3f33f8156bbc41968bef95bf0765d392 (127.11.223.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.924973 18825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.925067 18825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:56.925177 18825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.925242 18825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:56.925671 19013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:56.925786 19013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:56:56.931291 18839 master.cc:552] Master@127.11.223.188:42685 shutdown complete.
I20260621 07:56:56.931342 18839 master_runner.cc:305] Clearing existing system tablet
I20260621 07:56:56.931561 18839 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.931612 18839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.932890 18839 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260621 07:56:56.933734 19015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.933981 18839 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.934255 18839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:56.934360 18839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
uuid: "5a30916872334f0aa08d515def96dc57"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "1a2b5650c37c8c119cd44a79a8292429"
server_key_iv: "62f19ce9f65ce35ec8171b10195659c5"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.936733 18839 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:56.938920 18839 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260621 07:56:56.939070 18839 log.cc:1196] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260621 07:56:56.939363 18839 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Deleting consensus metadata
I20260621 07:56:56.939551 18839 master_runner.cc:315] Copying system tablet from 127.11.223.190:45495
I20260621 07:56:56.940683 18839 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: tablet copy: Beginning tablet copy session from remote peer at address 127.11.223.190:45495
I20260621 07:56:56.949892 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.941294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34634 (local address 127.11.223.190:45495)
0621 07:56:56.941471 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:56.941476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:56.941682 (+ 206us) server_negotiation.cc:407] Connection header received
0621 07:56:56.941809 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:56.941812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:56.941864 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:56.941952 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:56.942462 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.943246 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.944452 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.944796 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.944869 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.946254 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:56.946258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:56.946261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:56.946286 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:56.948196 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.948681 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.948685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.948687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.948775 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:56.949058 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:56.949061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:56.949063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:56.949260 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:56.949356 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:56.949653 (+ 297us) server_negotiation.cc:299] Negotiation successful
0621 07:56:56.949760 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260621 07:56:56.949911 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:56.941215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:34634)
0621 07:56:56.941585 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:56.941598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:56.941706 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:56.941997 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:56.942000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:56.942204 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:56.942337 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.942343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.943359 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.943363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.944275 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:56.944285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:56.945014 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:56.945017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:56.945114 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:56.945675 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:56.945709 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:56.946111 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:56.948302 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.948306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.948308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.948579 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.948871 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:56.948874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:56.948876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:56.948966 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:56.949357 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:56.949360 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:56.949451 (+ 91us) client_negotiation.cc:769] Sending connection context
0621 07:56:56.949557 (+ 106us) client_negotiation.cc:240] Negotiation successful
0621 07:56:56.949711 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":82,"threads_started":1}
I20260621 07:56:56.950546 18286 tablet_copy_service.cc:140] P c46deeb03ea2484491608fb69140b413: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5a30916872334f0aa08d515def96dc57 ({username='oryx', principal='oryx/127.11.223.188@KRBTEST.COM'} at 127.0.0.1:34634)
I20260621 07:56:56.950647 18286 tablet_copy_service.cc:161] P c46deeb03ea2484491608fb69140b413: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5a30916872334f0aa08d515def96dc57 at {username='oryx', principal='oryx/127.11.223.188@KRBTEST.COM'} at 127.0.0.1:34634: session id = 5a30916872334f0aa08d515def96dc57-00000000000000000000000000000000
I20260621 07:56:56.951661 18286 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Tablet Copy: opened 0 blocks and 1 log segments
I20260621 07:56:56.952898 18839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260621 07:56:56.955014 18839 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: tablet copy: Starting download of 0 data blocks...
I20260621 07:56:56.955245 18839 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: tablet copy: Starting download of 1 WAL segments...
I20260621 07:56:56.956768 18839 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260621 07:56:56.958498 18286 tablet_copy_service.cc:342] P c46deeb03ea2484491608fb69140b413: Request end of tablet copy session 5a30916872334f0aa08d515def96dc57-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.223.188@KRBTEST.COM'} at 127.0.0.1:34634
I20260621 07:56:56.958566 18286 tablet_copy_service.cc:433] P c46deeb03ea2484491608fb69140b413: ending tablet copy session 5a30916872334f0aa08d515def96dc57-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5a30916872334f0aa08d515def96dc57
I20260621 07:56:56.965075 18839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:56.966578 19022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:56.966615 19023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.969707 18839 server_base.cc:1034] running on GCE node
W20260621 07:56:56.970160 19025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:56.970415 18839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:56.970741 18839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:56.971887 18839 hybrid_clock.cc:648] HybridClock initialized: now 1782028616971874 us; error 27 us; skew 500 ppm
Jun 21 07:56:56 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:56.973476 18839 init.cc:377] Logged in from keytab as oryx/127.11.223.188@KRBTEST.COM (short username oryx)
I20260621 07:56:56.973892 18839 webserver.cc:460] Webserver started at http://127.11.223.188:44793/ using document root <none> and password file <none>
I20260621 07:56:56.974056 18839 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:56.974085 18839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:56.975234 18839 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260621 07:56:56.976233 19032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:56.976562 18839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260621 07:56:56.976637 18839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
uuid: "5a30916872334f0aa08d515def96dc57"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "1a2b5650c37c8c119cd44a79a8292429"
server_key_iv: "62f19ce9f65ce35ec8171b10195659c5"
server_key_version: "encryptionkey@0"
I20260621 07:56:56.976706 18839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:56.985193 18839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:56.987282 18839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:56.992231 18839 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.188:42685
I20260621 07:56:56.992336 19084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.188:42685 every 8 connection(s)
I20260621 07:56:56.992902 18839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
I20260621 07:56:56.994635 19085 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:56:56.995635 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Bootstrap starting.
I20260621 07:56:57.006698 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:56:57.007095 19085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: Bootstrap complete.
I20260621 07:56:57.007625 19085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } }
I20260621 07:56:57.007772 19085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5a30916872334f0aa08d515def96dc57, State: Initialized, Role: LEARNER
I20260621 07:56:57.007916 19085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } }
I20260621 07:56:57.008262 19088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:57.008415 19085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:56:57.008591 19088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: This master's current role is: LEARNER
I20260621 07:56:57.013484 19099 catalog_manager.cc:1259] Loaded cluster ID: dfbc6cceb4724d06a63caf4c47b2f307
I20260621 07:56:57.013592 19099 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: loading cluster ID for follower catalog manager: success
I20260621 07:56:57.015769 19099 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: acquiring CA information for follower catalog manager: success
I20260621 07:56:57.017244 19099 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260621 07:56:57.021695 19011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:57.021938 19011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:57.022020 19011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:57.026149 19011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:57.026222 19011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:57.026324 19011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:56:57.031519 19011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:45175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=33425
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:57.032512 19011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:57.033535 19011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:57.042582 19107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:57.042538 19105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:57.042546 19104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:57.042641 19011 server_base.cc:1034] running on GCE node
I20260621 07:56:57.043190 19011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:57.043709 19011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:57.044865 19011 hybrid_clock.cc:648] HybridClock initialized: now 1782028617044838 us; error 38 us; skew 500 ppm
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:57.047534 19011 init.cc:377] Logged in from keytab as oryx/127.11.223.129@KRBTEST.COM (short username oryx)
I20260621 07:56:57.048297 19011 webserver.cc:460] Webserver started at http://127.11.223.129:33425/ using document root <none> and password file <none>
I20260621 07:56:57.048906 19011 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:57.049007 19011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:57.052637 19011 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260621 07:56:57.054649 19114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.055840 19011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:57.056000 19011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "668417c1c7a7417fbfd08bc456a59488"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "600418a50eb9467e05a40251ca03c65d"
server_key_iv: "c3c01a70f6fb00d104fa0fdf09b31d8b"
server_key_version: "encryptionkey@0"
I20260621 07:56:57.056562 19011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:57.075067 18591 heartbeater.cc:503] Master 127.11.223.190:45495 was elected leader, sending a full tablet report...
I20260621 07:56:57.091318 19011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:57.094398 19011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:57.094588 19011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:57.095171 19011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:57.096097 19011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:57.096175 19011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.096231 19011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:57.096273 19011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.106242 19011 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:45175
I20260621 07:56:57.106312 19227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:45175 every 8 connection(s)
I20260621 07:56:57.107273 19011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:56:57.115955 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 19011
I20260621 07:56:57.116165 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18462
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.188@KRBTEST.COM
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
I20260621 07:56:57.123181 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.110928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:36037 (local address 127.11.223.188:42685)
0621 07:56:57.111183 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.111194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.111209 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:56:57.111659 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.111662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.111717 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.111824 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.113417 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.114482 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.115579 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.115942 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.116048 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.119020 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.119023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.119025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.119048 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.120887 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.121341 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.121347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.121352 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.121445 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.121925 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.121931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.121936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.122184 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.122300 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.122768 (+ 468us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.122911 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260621 07:56:57.123795 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.110817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.11.223.129:36037)
0621 07:56:57.111088 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.111105 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.111434 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.111989 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.111996 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.112679 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.113257 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.113266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.114595 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.114598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.115459 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.115466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.116058 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.116062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.116167 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.116585 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.116608 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.118843 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.120997 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.121001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.121011 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.121242 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.121564 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.121567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.121568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.121763 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.122304 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.122309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.122508 (+ 199us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.122682 (+ 174us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.122972 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":87,"spinlock_wait_cycles":10624,"thread_start_us":103,"threads_started":1}
I20260621 07:56:57.124977 19228 heartbeater.cc:347] Connected to a master server at 127.11.223.188:42685
I20260621 07:56:57.125211 19228 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.125716 19228 heartbeater.cc:511] Master 127.11.223.188:42685 requested a full tablet report, sending...
I20260621 07:56:57.125975 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:42267
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:45495
--webserver_port=42487
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:56:57.127346 19049 ts_manager.cc:194] Registered new tserver with Master: 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175)
I20260621 07:56:57.129273 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.111781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:57445 (local address 127.11.223.189:43445)
0621 07:56:57.111914 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.111919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.111931 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:56:57.111971 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.111973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.112017 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.112097 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.113416 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.114205 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.115422 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.115779 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.115848 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.119949 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.119952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.119954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.119975 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.126452 (+ 6477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.127805 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.127811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.127816 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.127912 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.128271 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.128274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.128275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.128519 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.128602 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.128999 (+ 397us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.129107 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:56:57.129590 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.111184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.11.223.129:57445)
0621 07:56:57.111404 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.111417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.111497 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.112126 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.112131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.112704 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.113257 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.113266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.114342 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.114345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.115293 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.115302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.116878 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.116881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.116953 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.117413 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.117430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.119777 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.127421 (+ 7644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.127425 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.127427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.127674 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.128039 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.128043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.128047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.128153 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.128682 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.128686 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.128790 (+ 104us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.128944 (+ 154us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.129109 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":142,"spinlock_wait_cycles":15488,"thread_start_us":99,"threads_started":1}
I20260621 07:56:57.130224 19230 heartbeater.cc:347] Connected to a master server at 127.11.223.189:43445
I20260621 07:56:57.130362 19230 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.130529 19230 heartbeater.cc:511] Master 127.11.223.189:43445 requested a full tablet report, sending...
I20260621 07:56:57.131035 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.109465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:43017)
0621 07:56:57.110386 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.110443 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.111434 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.111990 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.111998 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.112677 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.113242 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.113255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.114056 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.114059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.114915 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.114924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.115410 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.115414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.115584 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.116455 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.116507 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.126665 (+ 10158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.128930 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.128936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.128941 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.129277 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.129873 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.129876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.129878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.129979 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.130489 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.130492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.130568 (+ 76us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.130732 (+ 164us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.130876 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":90,"thread_start_us":577,"threads_started":1}
I20260621 07:56:57.131178 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.108983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:43017 (local address 127.11.223.190:45495)
0621 07:56:57.109196 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.109206 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.110542 (+ 1336us) server_negotiation.cc:407] Connection header received
0621 07:56:57.111677 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.111680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.111737 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.111793 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.113389 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.113921 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.115037 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.115269 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.115381 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.126802 (+ 11421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.126806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.126808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.126834 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.128812 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.129611 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.129629 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.129631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.129731 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.130123 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.130128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.130130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.130389 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.130507 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.130820 (+ 313us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.130959 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260621 07:56:57.131603 18867 ts_manager.cc:194] Registered new tserver with Master: 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175)
I20260621 07:56:57.132184 19229 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45495
I20260621 07:56:57.132275 19229 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.132506 19229 heartbeater.cc:511] Master 127.11.223.190:45495 requested a full tablet report, sending...
I20260621 07:56:57.133348 18266 ts_manager.cc:194] Re-registered known tserver with Master: 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175)
I20260621 07:56:57.134004 18266 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:43017
W20260621 07:56:57.238358 19236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:57.238591 19236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:57.238685 19236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:57.242667 19236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:57.242722 19236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:57.242816 19236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:56:57.247951 19236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:42267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=42487
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:57.248934 19236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:57.249930 19236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:57.259147 19244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:57.259255 19236 server_base.cc:1034] running on GCE node
W20260621 07:56:57.259202 19242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:57.259183 19241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:57.259897 19236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:57.260586 19236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:57.261766 19236 hybrid_clock.cc:648] HybridClock initialized: now 1782028617261733 us; error 50 us; skew 500 ppm
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:57.264534 19236 init.cc:377] Logged in from keytab as oryx/127.11.223.130@KRBTEST.COM (short username oryx)
I20260621 07:56:57.265290 19236 webserver.cc:460] Webserver started at http://127.11.223.130:42487/ using document root <none> and password file <none>
I20260621 07:56:57.265843 19236 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:57.265944 19236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:57.269307 19236 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260621 07:56:57.271193 19251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.272393 19236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260621 07:56:57.272500 19236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "88efbf17dec149638cba8a3b8fc78145"
format_stamp: "Formatted at 2026-06-21 07:56:55 on dist-test-slave-0pn3"
server_key: "0a30ff7aefe2c818ef02de0bebbdc9a9"
server_key_iv: "9d740a71ac05f9daf084e0006984e434"
server_key_version: "encryptionkey@0"
I20260621 07:56:57.273021 19236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:57.278834 18725 heartbeater.cc:503] Master 127.11.223.190:45495 was elected leader, sending a full tablet report...
I20260621 07:56:57.300110 19236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:57.303372 19236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:57.303563 19236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:57.304128 19236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:57.304947 19236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:57.305022 19236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.305089 19236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:57.305142 19236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.314291 19236 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:42267
I20260621 07:56:57.314313 19364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:42267 every 8 connection(s)
I20260621 07:56:57.315212 19236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:56:57.322846 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 19236
I20260621 07:56:57.323020 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18596
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
I20260621 07:56:57.330229 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:45215
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:45495
--webserver_port=36585
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:56:57.330284 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.316939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:52941 (local address 127.11.223.190:45495)
0621 07:56:57.317065 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.317068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.318186 (+ 1118us) server_negotiation.cc:407] Connection header received
0621 07:56:57.319412 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.319416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.319469 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.319543 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.320980 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.321765 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.322854 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.323218 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.323321 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.326350 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.326356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.326362 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.326392 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.328101 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.328619 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.328624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.328626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.328704 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.329067 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.329071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.329072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.329251 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.329352 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.329980 (+ 628us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.330091 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:56:57.331841 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.317238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:52941)
0621 07:56:57.318027 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.318067 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.319086 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.319584 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.319588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.320200 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.320824 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.320831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.321901 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.321904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.322735 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.322744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.323512 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.323518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.323738 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.324371 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.324395 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.326193 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.328213 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.328217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.328227 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.328519 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.328816 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.328819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.328821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.328967 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.329349 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.329355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.329681 (+ 326us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.330459 (+ 778us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.330791 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":255,"spinlock_wait_cycles":50176,"thread_start_us":495,"threads_started":1}
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.188@KRBTEST.COM
I20260621 07:56:57.333022 19366 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45495
I20260621 07:56:57.333228 19366 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.333676 19366 heartbeater.cc:511] Master 127.11.223.190:45495 requested a full tablet report, sending...
I20260621 07:56:57.334810 18266 ts_manager.cc:194] Re-registered known tserver with Master: 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267)
I20260621 07:56:57.335528 18266 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:52941
I20260621 07:56:57.336279 19370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.318420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.11.223.130:36273)
0621 07:56:57.318709 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.318725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.319092 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.319576 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.319585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.320200 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.320822 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.320831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.321618 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.321621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.322882 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.322894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.323851 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.323858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.323982 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.328860 (+ 4878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.328879 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.330831 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.334321 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.334324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.334326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.334559 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.334868 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.334872 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.334874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.334960 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.335484 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.335489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.335571 (+ 82us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.335658 (+ 87us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.335790 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":191,"spinlock_wait_cycles":44288,"thread_start_us":104,"threads_started":1}
I20260621 07:56:57.336539 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.318902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:44811 (local address 127.11.223.188:42685)
0621 07:56:57.319020 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.319024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.319059 (+ 35us) server_negotiation.cc:407] Connection header received
0621 07:56:57.319550 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.319556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.319604 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.319694 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.320941 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.321771 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.329986 (+ 8215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.330376 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.330470 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.332906 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.332911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.332913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.332946 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.334708 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.335164 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.335168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.335169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.335240 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.335519 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.335522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.335524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.335717 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.335787 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.336221 (+ 434us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.336380 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:56:57.336774 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.318537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:36273 (local address 127.11.223.189:43445)
0621 07:56:57.318661 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.318665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.318793 (+ 128us) server_negotiation.cc:407] Connection header received
0621 07:56:57.319290 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.319293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.319347 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.319419 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.320962 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.321482 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.323022 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.323409 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.323514 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.330992 (+ 7478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.330996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.330998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.331023 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.332906 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.334659 (+ 1753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.334664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.334666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.334791 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.335093 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.335102 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.335103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.335365 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.335445 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.336506 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.336618 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:56:57.336987 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.318797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.11.223.130:44811)
0621 07:56:57.318988 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.319001 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.319091 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.319706 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.319709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.320202 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.320801 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.320814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.321880 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.321883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.329827 (+ 7944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.329841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.330636 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.330642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.330764 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.331265 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.331284 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.332747 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.334838 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.334841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.334843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.335062 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.335346 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.335348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.335350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.335430 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.335921 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.335924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.336008 (+ 84us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.336707 (+ 699us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.336827 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":114,"spinlock_wait_cycles":40320,"thread_start_us":78,"threads_started":1}
I20260621 07:56:57.338932 19367 heartbeater.cc:347] Connected to a master server at 127.11.223.189:43445
I20260621 07:56:57.338970 19365 heartbeater.cc:347] Connected to a master server at 127.11.223.188:42685
I20260621 07:56:57.339010 19367 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.339059 19365 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.339150 19367 heartbeater.cc:511] Master 127.11.223.189:43445 requested a full tablet report, sending...
I20260621 07:56:57.339222 19365 heartbeater.cc:511] Master 127.11.223.188:42685 requested a full tablet report, sending...
I20260621 07:56:57.339818 18867 ts_manager.cc:194] Registered new tserver with Master: 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267)
I20260621 07:56:57.339983 19049 ts_manager.cc:194] Registered new tserver with Master: 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267)
W20260621 07:56:57.440114 19372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:56:57.440377 19372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:56:57.440448 19372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:56:57.444309 19372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:56:57.444432 19372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:56:57.444522 19372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:56:57.449397 19372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:45215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=36585
--tserver_master_addrs=127.11.223.190:45495,127.11.223.189:43445,127.11.223.188:42685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:56:57.450309 19372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:56:57.451200 19372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:56:57.459856 19377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:56:57.459864 19378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:57.459918 19372 server_base.cc:1034] running on GCE node
W20260621 07:56:57.459864 19380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:56:57.460594 19372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:56:57.461206 19372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:56:57.462360 19372 hybrid_clock.cc:648] HybridClock initialized: now 1782028617462344 us; error 27 us; skew 500 ppm
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:56:57.464824 19372 init.cc:377] Logged in from keytab as oryx/127.11.223.131@KRBTEST.COM (short username oryx)
I20260621 07:56:57.465556 19372 webserver.cc:460] Webserver started at http://127.11.223.131:36585/ using document root <none> and password file <none>
I20260621 07:56:57.466068 19372 fs_manager.cc:359] Metadata directory not provided
I20260621 07:56:57.466181 19372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:56:57.469919 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.466520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:49016)
0621 07:56:57.466736 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.466747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.466825 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.467070 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.467072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.467084 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:57.467168 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.467180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.468071 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.468073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.468826 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.468835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.469552 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.469556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.469657 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.469682 (+ 25us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.469749 (+ 67us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.469805 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":58,"threads_started":1}
I20260621 07:56:57.469942 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.466594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49016 (local address 127.11.223.188:42685)
0621 07:56:57.466774 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.466778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.466794 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:56:57.466906 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.466909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.466967 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.467057 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:57.467271 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.467979 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.468941 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.469435 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.469551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.469698 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.469768 (+ 70us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.469827 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 07:56:57.470270 19372 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:56:57.470666 19059 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260621 07:56:57.471016 19088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c46deeb03ea2484491608fb69140b413. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:57.471156 19088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: This master's current role is: LEARNER
I20260621 07:56:57.472409 19388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.473441 19372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:56:57.473558 19372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "9b442adc28634d16bf30af967fe1c4ea"
format_stamp: "Formatted at 2026-06-21 07:56:56 on dist-test-slave-0pn3"
server_key: "0532838531d214cb0220951413ce8df1"
server_key_iv: "df8545e7f47354f37e8b767afbd739cf"
server_key_version: "encryptionkey@0"
I20260621 07:56:57.474043 19372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:56:57.489984 19372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:56:57.492394 19372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:56:57.492578 19372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:56:57.493041 19372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:56:57.493736 19372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:56:57.493801 19372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.493860 19372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:56:57.493901 19372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:56:57.502756 19372 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:45215
I20260621 07:56:57.502796 19501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:45215 every 8 connection(s)
I20260621 07:56:57.503598 19372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:56:57.508384 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 19372
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.189@KRBTEST.COM
Jun 21 07:56:57 dist-test-slave-0pn3 krb5kdc[18221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.188@KRBTEST.COM
I20260621 07:56:57.522068 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.507013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:41871 (local address 127.11.223.189:43445)
0621 07:56:57.507145 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.507149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.507190 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:56:57.507655 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.507657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.507723 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.507798 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.509188 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.509929 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.511844 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.512159 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.512248 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.516259 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.516262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.516266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.516292 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.520150 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.520637 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.520644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.520645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.520742 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.521040 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.521043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.521045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.521234 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.521316 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.521820 (+ 504us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.521924 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 07:56:57.522600 19508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.506914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.11.223.131:41871)
0621 07:56:57.507121 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.507134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.507225 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.507843 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.507847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.508302 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.508971 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.508981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.510135 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.510137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.511501 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.511511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.512437 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.512441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.512651 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.514454 (+ 1803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.514466 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.516077 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.520267 (+ 4190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.520271 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.520276 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.520520 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.520854 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.520857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.520859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.520940 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.521356 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.521358 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.521441 (+ 83us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.521550 (+ 109us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.521652 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":107,"spinlock_wait_cycles":16000,"thread_start_us":78,"threads_started":1}
I20260621 07:56:57.523074 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.506664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:42989 (local address 127.11.223.188:42685)
0621 07:56:57.506794 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.506800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.506908 (+ 108us) server_negotiation.cc:407] Connection header received
0621 07:56:57.507579 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.507583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.507628 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.507729 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.509120 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.509907 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.513623 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.513941 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.514038 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.516721 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.516726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.516731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.516758 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.518868 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.519812 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.519816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.519818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.519902 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.520293 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.520296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.520298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.520539 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.520627 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.521854 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.522927 (+ 1073us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260621 07:56:57.523120 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.511674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.11.223.190:45495)
0621 07:56:57.511882 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.511886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.511897 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:56:57.511925 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.511928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.511961 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.512940 (+ 979us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.513065 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.513775 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.514947 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.515297 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.515762 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.517613 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.517619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.517625 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.517675 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.520838 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.521324 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.521328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.521331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.521420 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.521792 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.521795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.521797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.522003 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.522735 (+ 732us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.522819 (+ 84us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.522939 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260621 07:56:57.523236 19507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.506572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.11.223.131:42989)
0621 07:56:57.506824 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.506838 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.507236 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.507743 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.507746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.508302 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.508952 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.508970 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.510058 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.510061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.513493 (+ 3432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.513502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.514164 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.514168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.514247 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.515022 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.515035 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.516557 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.519351 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.519353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.519356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.519675 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.520009 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.520013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.520015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.520188 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.520674 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.520680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.520953 (+ 273us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.521121 (+ 168us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.521419 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":2334,"thread_start_us":83,"threads_started":1}
I20260621 07:56:57.524253 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.511628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.11.223.188:42685)
0621 07:56:57.512485 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.512490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.512504 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:56:57.512551 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.512554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.512591 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.514074 (+ 1483us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.514159 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.515192 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.517741 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.518089 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.518210 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.520411 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.520416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.520422 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.520447 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.522232 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.522737 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.522741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.522743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.522820 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.523109 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.523112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.523113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.523332 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.523905 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.523975 (+ 70us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.524083 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":77,"threads_started":1}
I20260621 07:56:57.524827 19504 heartbeater.cc:347] Connected to a master server at 127.11.223.189:43445
I20260621 07:56:57.525020 19504 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.525048 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.505075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:50557 (local address 127.11.223.190:45495)
0621 07:56:57.505198 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.505201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.506334 (+ 1133us) server_negotiation.cc:407] Connection header received
0621 07:56:57.507422 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.507425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.507487 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.507558 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.510463 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.511200 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.512524 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.512855 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.512939 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.517491 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.517494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.517497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.517523 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.519155 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.523739 (+ 4584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.523742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.523743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.523826 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.524140 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.524161 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.524162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.524399 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.524488 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.524808 (+ 320us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.524918 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":55}
I20260621 07:56:57.525429 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.505414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:50557)
0621 07:56:57.506198 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.506234 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.507219 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.507722 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.507730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.510253 (+ 2523us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:56:57.510346 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.510351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.511524 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.511530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.512399 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.512408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.513111 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.513118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.513239 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.513736 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:56:57.513758 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:56:57.517327 (+ 3569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:56:57.519266 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.519271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.519282 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.523632 (+ 4350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.523931 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:56:57.523952 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:56:57.523957 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:56:57.524043 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:56:57.524542 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:56:57.524545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:56:57.524624 (+ 79us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.525145 (+ 521us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.525284 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":3,"thread_start_us":488,"threads_started":1}
I20260621 07:56:57.525473 19504 heartbeater.cc:511] Master 127.11.223.189:43445 requested a full tablet report, sending...
I20260621 07:56:57.525892 19502 heartbeater.cc:347] Connected to a master server at 127.11.223.188:42685
I20260621 07:56:57.525954 19502 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.526074 19502 heartbeater.cc:511] Master 127.11.223.188:42685 requested a full tablet report, sending...
I20260621 07:56:57.526619 19049 ts_manager.cc:194] Registered new tserver with Master: 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215)
I20260621 07:56:57.528623 18867 ts_manager.cc:194] Registered new tserver with Master: 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215)
I20260621 07:56:57.528857 19503 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45495
I20260621 07:56:57.528932 19503 heartbeater.cc:464] Registering TS with master...
I20260621 07:56:57.526341 18266 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260621 07:56:57.529155 19503 heartbeater.cc:511] Master 127.11.223.190:45495 requested a full tablet report, sending...
I20260621 07:56:57.529742 18265 ts_manager.cc:194] Re-registered known tserver with Master: 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215)
W20260621 07:56:57.530256 18266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:56:57.530278 18265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:50557
I20260621 07:56:57.532364 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.511352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.11.223.189:43445)
0621 07:56:57.518431 (+ 7079us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.518436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.518449 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:56:57.518482 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.518487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.518542 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.518627 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.525564 (+ 6937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.526218 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.527195 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.527439 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.527510 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.528886 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.528889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.528891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.528912 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.530561 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.531081 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.531084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.531086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.531177 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.531489 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.531492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.531494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.531676 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.531797 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.532025 (+ 228us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.532126 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6991,"thread_start_us":86,"threads_started":1}
I20260621 07:56:57.533079 18824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: NON_VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: true } } }
I20260621 07:56:57.533209 18824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:57.536182 19520 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: attempting to promote NON_VOTER 5a30916872334f0aa08d515def96dc57 to VOTER
I20260621 07:56:57.536538 19520 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:56:57.539778 19059 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 LEARNER]: Refusing update from remote peer c46deeb03ea2484491608fb69140b413: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260621 07:56:57.539844 18876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Refusing update from remote peer c46deeb03ea2484491608fb69140b413: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260621 07:56:57.540248 19521 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260621 07:56:57.540452 18824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260621 07:56:57.542106 18824 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5a30916872334f0aa08d515def96dc57 (127.11.223.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.542502 18876 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5a30916872334f0aa08d515def96dc57 (127.11.223.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.542865 19059 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5a30916872334f0aa08d515def96dc57 (127.11.223.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.543547 19520 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.543583 18824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.543653 19520 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:57.543673 18824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:57.543766 18824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.543824 18824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413 [sys.catalog]: This master's current role is: LEADER
I20260621 07:56:57.544572 19088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.544705 19088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:56:57.544797 19529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c46deeb03ea2484491608fb69140b413" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:56:57.544951 19529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:56:57.551034 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.545977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:42267 (local address 127.0.0.1:35248)
0621 07:56:57.546083 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.546097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.546175 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.547015 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.547018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.547031 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:57.547111 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.547116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.548450 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.548455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.549501 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.549510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.550604 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.550608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.550712 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.550745 (+ 33us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.550820 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.550927 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":2}
I20260621 07:56:57.551113 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.546163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35248 (local address 127.11.223.130:42267)
0621 07:56:57.546496 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.546506 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.546527 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:56:57.546596 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.546608 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.546844 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.547048 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:57.547247 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.548288 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.549631 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.550466 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.550617 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.550734 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.550858 (+ 124us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.550961 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":121,"threads_started":1}
I20260621 07:56:57.552240 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.547131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:45175 (local address 127.0.0.1:54512)
0621 07:56:57.547504 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.547515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.547582 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.548256 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.548259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.548274 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:57.548393 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.548399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.549793 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.549796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.550955 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.550964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.551904 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.551907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.551984 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.552005 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.552075 (+ 70us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.552132 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":74,"threads_started":1}
I20260621 07:56:57.552302 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.547489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54512 (local address 127.11.223.129:45175)
0621 07:56:57.547808 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.547821 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.547844 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:56:57.547915 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.547934 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.548101 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.548529 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:57.548651 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.549654 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.551078 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.551769 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.551926 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.552029 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.552102 (+ 73us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.552180 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":113,"threads_started":1}
I20260621 07:56:57.553331 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.546541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45215 (local address 127.0.0.1:46894)
0621 07:56:57.546774 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.546787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.546864 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.548246 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.548249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.548267 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:57.548378 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.548385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.550799 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.550803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.551735 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.551749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.552967 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.552971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.553053 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.553074 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.553150 (+ 76us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.553217 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":36,"thread_start_us":85,"threads_started":1}
I20260621 07:56:57.553516 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.546681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.11.223.131:45215)
0621 07:56:57.547914 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.547936 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.547952 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:56:57.548000 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.548003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.548124 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.548254 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:57.549646 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.550666 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.551853 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.552841 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.552968 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.553071 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.553174 (+ 103us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.553256 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":128,"threads_started":1}
I20260621 07:56:57.553974 19299 tablet_service.cc:1467] Processing CreateTablet for tablet 2f790babf457418c80305f1bf75e8199 (DEFAULT_TABLE table=test-table [id=a12bb7901dc44bdb93087cf26a7b9cca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:57.554979 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f790babf457418c80305f1bf75e8199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:57.554739 19162 tablet_service.cc:1467] Processing CreateTablet for tablet 2f790babf457418c80305f1bf75e8199 (DEFAULT_TABLE table=test-table [id=a12bb7901dc44bdb93087cf26a7b9cca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:57.555586 19162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f790babf457418c80305f1bf75e8199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:57.555435 19436 tablet_service.cc:1467] Processing CreateTablet for tablet 2f790babf457418c80305f1bf75e8199 (DEFAULT_TABLE table=test-table [id=a12bb7901dc44bdb93087cf26a7b9cca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:56:57.556044 19436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f790babf457418c80305f1bf75e8199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:56:57.561968 19535 tablet_bootstrap.cc:492] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Bootstrap starting.
I20260621 07:56:57.563144 19536 tablet_bootstrap.cc:492] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Bootstrap starting.
I20260621 07:56:57.563845 19535 tablet_bootstrap.cc:654] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:57.564630 19537 tablet_bootstrap.cc:492] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Bootstrap starting.
I20260621 07:56:57.564761 19535 log.cc:824] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:57.564941 19536 tablet_bootstrap.cc:654] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:57.565706 19536 log.cc:824] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:57.566514 19537 tablet_bootstrap.cc:654] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Neither blocks nor log segments found. Creating new log.
I20260621 07:56:57.566614 19535 tablet_bootstrap.cc:492] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: No bootstrap required, opened a new log
I20260621 07:56:57.566781 19535 ts_tablet_manager.cc:1397] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 07:56:57.567162 19537 log.cc:824] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Log is configured to *not* fsync() on all Append() calls
I20260621 07:56:57.567441 19536 tablet_bootstrap.cc:492] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: No bootstrap required, opened a new log
I20260621 07:56:57.567605 19536 ts_tablet_manager.cc:1397] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 07:56:57.568749 19537 tablet_bootstrap.cc:492] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: No bootstrap required, opened a new log
I20260621 07:56:57.568919 19537 ts_tablet_manager.cc:1397] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260621 07:56:57.571105 19535 raft_consensus.cc:348] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.571376 19535 raft_consensus.cc:374] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:57.571436 19535 raft_consensus.cc:729] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b442adc28634d16bf30af967fe1c4ea, State: Initialized, Role: FOLLOWER
I20260621 07:56:57.571916 19535 consensus_queue.cc:260] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.571908 19536 raft_consensus.cc:348] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.572127 19536 raft_consensus.cc:374] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:57.572197 19536 raft_consensus.cc:729] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668417c1c7a7417fbfd08bc456a59488, State: Initialized, Role: FOLLOWER
I20260621 07:56:57.572495 19537 raft_consensus.cc:348] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.572597 19536 consensus_queue.cc:260] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.572714 19537 raft_consensus.cc:374] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:56:57.572762 19537 raft_consensus.cc:729] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88efbf17dec149638cba8a3b8fc78145, State: Initialized, Role: FOLLOWER
I20260621 07:56:57.572930 19535 ts_tablet_manager.cc:1428] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260621 07:56:57.573099 19503 heartbeater.cc:503] Master 127.11.223.190:45495 was elected leader, sending a full tablet report...
I20260621 07:56:57.573137 19537 consensus_queue.cc:260] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.573401 19536 ts_tablet_manager.cc:1428] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260621 07:56:57.574038 19229 heartbeater.cc:503] Master 127.11.223.190:45495 was elected leader, sending a full tablet report...
I20260621 07:56:57.577965 19537 ts_tablet_manager.cc:1428] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260621 07:56:57.578253 19366 heartbeater.cc:503] Master 127.11.223.190:45495 was elected leader, sending a full tablet report...
W20260621 07:56:57.609150 19231 tablet.cc:2369] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:56:57.755136 19505 tablet.cc:2369] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:57.772356 19542 raft_consensus.cc:482] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:56:57.772599 19542 raft_consensus.cc:504] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.773779 19542 leader_election.cc:288] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267), 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215)
I20260621 07:56:57.777750 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.774089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:36111 (local address 127.11.223.130:42267)
0621 07:56:57.774218 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.774223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.774357 (+ 134us) server_negotiation.cc:407] Connection header received
0621 07:56:57.774512 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.774515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.774562 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.774639 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:57.774963 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.775456 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.776648 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.777121 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.777213 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.777302 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.777569 (+ 267us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.777630 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:56:57.777769 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.774005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45215 (local address 127.11.223.129:50207)
0621 07:56:57.774358 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.774369 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.774474 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.774707 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.774710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.774726 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:57.774881 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.774885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.775650 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.775652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.776572 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.776581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.777433 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.777448 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.777557 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.777573 (+ 16us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.777626 (+ 53us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.777669 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":21,"thread_start_us":60,"threads_started":1}
I20260621 07:56:57.777872 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.774126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:50207 (local address 127.11.223.131:45215)
0621 07:56:57.774248 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.774272 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.774435 (+ 163us) server_negotiation.cc:407] Connection header received
0621 07:56:57.774578 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.774581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.774619 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.774697 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:57.775036 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.775516 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.776699 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.777215 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.777330 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.777383 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.777669 (+ 286us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.777740 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:56:57.777771 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.773970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:42267 (local address 127.11.223.129:36111)
0621 07:56:57.774275 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:57.774286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:57.774409 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:57.774658 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:57.774661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:57.774676 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:57.774858 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.774864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.775579 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.775584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.776539 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:57.776548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.777247 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.777254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:57.777424 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.777466 (+ 42us) client_negotiation.cc:769] Sending connection context
0621 07:56:57.777550 (+ 84us) client_negotiation.cc:240] Negotiation successful
0621 07:56:57.777623 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":40,"thread_start_us":91,"threads_started":1}
I20260621 07:56:57.778525 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f790babf457418c80305f1bf75e8199" candidate_uuid: "668417c1c7a7417fbfd08bc456a59488" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b442adc28634d16bf30af967fe1c4ea" is_pre_election: true
I20260621 07:56:57.778764 19319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f790babf457418c80305f1bf75e8199" candidate_uuid: "668417c1c7a7417fbfd08bc456a59488" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88efbf17dec149638cba8a3b8fc78145" is_pre_election: true
I20260621 07:56:57.778820 19456 raft_consensus.cc:2454] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668417c1c7a7417fbfd08bc456a59488 in term 0.
I20260621 07:56:57.779009 19319 raft_consensus.cc:2454] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668417c1c7a7417fbfd08bc456a59488 in term 0.
I20260621 07:56:57.779227 19115 leader_election.cc:302] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668417c1c7a7417fbfd08bc456a59488, 9b442adc28634d16bf30af967fe1c4ea; no voters:
I20260621 07:56:57.779492 19542 raft_consensus.cc:2781] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:56:57.779567 19542 raft_consensus.cc:482] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:56:57.779596 19542 raft_consensus.cc:3037] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:57.780761 19542 raft_consensus.cc:504] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.781078 19542 leader_election.cc:288] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [CANDIDATE]: Term 1 election: Requested vote from peers 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267), 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215)
I20260621 07:56:57.781375 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f790babf457418c80305f1bf75e8199" candidate_uuid: "668417c1c7a7417fbfd08bc456a59488" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b442adc28634d16bf30af967fe1c4ea"
I20260621 07:56:57.781394 19319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f790babf457418c80305f1bf75e8199" candidate_uuid: "668417c1c7a7417fbfd08bc456a59488" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88efbf17dec149638cba8a3b8fc78145"
I20260621 07:56:57.781489 19319 raft_consensus.cc:3037] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:57.781489 19456 raft_consensus.cc:3037] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:56:57.782727 19456 raft_consensus.cc:2454] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668417c1c7a7417fbfd08bc456a59488 in term 1.
I20260621 07:56:57.782727 19319 raft_consensus.cc:2454] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668417c1c7a7417fbfd08bc456a59488 in term 1.
I20260621 07:56:57.782996 19118 leader_election.cc:302] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668417c1c7a7417fbfd08bc456a59488, 88efbf17dec149638cba8a3b8fc78145; no voters:
I20260621 07:56:57.783143 19542 raft_consensus.cc:2781] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:56:57.783311 19542 raft_consensus.cc:686] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 1 LEADER]: Becoming Leader. State: Replica: 668417c1c7a7417fbfd08bc456a59488, State: Running, Role: LEADER
I20260621 07:56:57.783885 19542 consensus_queue.cc:237] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } }
I20260621 07:56:57.787159 18266 catalog_manager.cc:5495] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 reported cstate change: term changed from 0 to 1, leader changed from <none> to 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129). New cstate: current_term: 1 leader_uuid: "668417c1c7a7417fbfd08bc456a59488" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } health_report { overall_health: UNKNOWN } } }
W20260621 07:56:57.816691 19368 tablet.cc:2369] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:56:57.837839 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.834628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.11.223.129:45175)
0621 07:56:57.834783 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.834787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.834802 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:56:57.834880 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.834882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.834935 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.835076 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:56:57.835347 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.835814 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.836626 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.836845 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.836922 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.837150 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:56:57.837533 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:56:57.837628 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.837691 (+ 63us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.837742 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:56:57.843216 19319 raft_consensus.cc:1261] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Refusing update from remote peer 668417c1c7a7417fbfd08bc456a59488: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:57.843247 19456 raft_consensus.cc:1261] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Refusing update from remote peer 668417c1c7a7417fbfd08bc456a59488: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:56:57.843909 19546 consensus_queue.cc:1035] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:57.844066 19542 consensus_queue.cc:1035] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:56:57.851641 19549 mvcc.cc:205] Tried to move back new op lower bound from 7299189218679431168 to 7299189218445651968. Current Snapshot: MvccSnapshot[applied={T|T < 7299189218679431168}]
I20260621 07:56:57.852475 19551 mvcc.cc:205] Tried to move back new op lower bound from 7299189218679431168 to 7299189218445651968. Current Snapshot: MvccSnapshot[applied={T|T < 7299189218679431168}]
I20260621 07:56:57.859318 18265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:57.859474 18265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:56:57.863291 18265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c46deeb03ea2484491608fb69140b413: Sending DeleteTablet for 3 replicas of tablet 2f790babf457418c80305f1bf75e8199
I20260621 07:56:57.864557 19436 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f790babf457418c80305f1bf75e8199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:57 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:46894
I20260621 07:56:57.864677 19299 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f790babf457418c80305f1bf75e8199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:57 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35248
I20260621 07:56:57.864974 19559 tablet_replica.cc:330] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: stopping tablet replica
I20260621 07:56:57.865293 19559 raft_consensus.cc:2229] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:57.865617 19559 raft_consensus.cc:2258] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:57.866199 19162 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f790babf457418c80305f1bf75e8199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:56:57 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:54512
I20260621 07:56:57.866671 19562 tablet_replica.cc:330] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: stopping tablet replica
I20260621 07:56:57.866964 19562 raft_consensus.cc:2229] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:56:57.867306 19562 raft_consensus.cc:2258] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:57.867830 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18234
I20260621 07:56:57.868727 19559 ts_tablet_manager.cc:1905] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:57.869037 19562 ts_tablet_manager.cc:1905] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:57.871418 19559 ts_tablet_manager.cc:1918] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:57.871512 19559 log.cc:1196] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/2f790babf457418c80305f1bf75e8199
I20260621 07:56:57.871799 19562 ts_tablet_manager.cc:1918] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:57.871851 19559 ts_tablet_manager.cc:1939] T 2f790babf457418c80305f1bf75e8199 P 9b442adc28634d16bf30af967fe1c4ea: Deleting consensus metadata
I20260621 07:56:57.871959 19562 log.cc:1196] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/2f790babf457418c80305f1bf75e8199
I20260621 07:56:57.872298 19562 ts_tablet_manager.cc:1939] T 2f790babf457418c80305f1bf75e8199 P 668417c1c7a7417fbfd08bc456a59488: Deleting consensus metadata
I20260621 07:56:57.875916 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.865568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.11.223.189:43445)
0621 07:56:57.865682 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.865688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.865922 (+ 234us) server_negotiation.cc:407] Connection header received
0621 07:56:57.866087 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.866090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.866136 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.866268 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.867160 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.867850 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.869486 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.869973 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.870066 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.872270 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.872276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.872282 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.872336 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.874083 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.874603 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.874606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.874608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.874700 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.875026 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.875029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.875031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.875255 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.875381 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.875653 (+ 272us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.875765 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260621 07:56:57.876143 19115 connection.cc:535] server connection from 127.0.0.1:54512 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260621 07:56:57.876513 19390 connection.cc:535] server connection from 127.0.0.1:46894 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260621 07:56:57.879473 19560 tablet_replica.cc:330] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: stopping tablet replica
I20260621 07:56:57.879806 19560 raft_consensus.cc:2229] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:56:57.880102 19560 raft_consensus.cc:2258] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:56:57.881902 19560 ts_tablet_manager.cc:1905] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:56:57.884827 19560 ts_tablet_manager.cc:1918] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:56:57.884930 19560 log.cc:1196] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/2f790babf457418c80305f1bf75e8199
I20260621 07:56:57.885289 19560 ts_tablet_manager.cc:1939] T 2f790babf457418c80305f1bf75e8199 P 88efbf17dec149638cba8a3b8fc78145: Deleting consensus metadata
I20260621 07:56:57.885882 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.866625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49028 (local address 127.11.223.188:42685)
0621 07:56:57.866748 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.866752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.876823 (+ 10071us) server_negotiation.cc:407] Connection header received
0621 07:56:57.877006 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.877012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.877071 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.877121 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.878048 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.878809 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.879756 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.880071 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.880177 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.881809 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.881812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.881815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.881837 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.883439 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.883915 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.883918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.883920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.883998 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.884285 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.884288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.884289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.884531 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.884662 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.885615 (+ 953us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.885738 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260621 07:56:57.886257 19252 connection.cc:441] server connection from 127.0.0.1:35248 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35248 (request call id 1) could send its response
I20260621 07:56:57.891219 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.879948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.11.223.189:43445)
0621 07:56:57.880048 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.880052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.881047 (+ 995us) server_negotiation.cc:407] Connection header received
0621 07:56:57.881203 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.881206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.881255 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.881301 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.882120 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.882911 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.884062 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.884428 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.884498 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.887510 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.887513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.887516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.887538 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.889104 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.889866 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.889870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.889871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.889958 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.890267 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.890272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.890275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.890501 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.890630 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.890952 (+ 322us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.891059 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 07:56:57.894349 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:57.880536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.11.223.188:42685)
0621 07:56:57.880654 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:57.880658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:57.885220 (+ 4562us) server_negotiation.cc:407] Connection header received
0621 07:56:57.886587 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:57.886590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:57.886629 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:57.886673 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:56:57.887413 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.888000 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.888964 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:57.889175 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:57.889288 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:57.890868 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:56:57.890872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:56:57.890874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:56:57.890911 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:56:57.892773 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.893196 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.893200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.893202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.893283 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:56:57.893553 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:56:57.893557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:56:57.893559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:56:57.893730 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:56:57.893859 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:57.894098 (+ 239us) server_negotiation.cc:299] Negotiation successful
0621 07:56:57.894202 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260621 07:56:58.856319 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:58.855856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:41255)
0621 07:56:58.856093 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:58.856202 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
W20260621 07:56:58.856690 19229 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:45495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:58.856897 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:58.856493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:59283)
0621 07:56:58.856711 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:58.856786 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":58,"threads_started":1}
W20260621 07:56:58.857168 19503 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:45495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:58.858829 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:58.858478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:38683)
0621 07:56:58.858692 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:58.858758 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":56,"threads_started":1}
W20260621 07:56:58.859083 19366 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:45495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:59.364106 19573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c46deeb03ea2484491608fb69140b413)
I20260621 07:56:59.364418 19573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:56:59.364471 19574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c46deeb03ea2484491608fb69140b413)
I20260621 07:56:59.364643 19574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:56:59.365468 19573 leader_election.cc:288] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495), 3f33f8156bbc41968bef95bf0765d392 (127.11.223.189:43445)
I20260621 07:56:59.365511 19574 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495), 5a30916872334f0aa08d515def96dc57 (127.11.223.188:42685)
I20260621 07:56:59.366040 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.365578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35128)
0621 07:56:59.365834 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.365920 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":60,"threads_started":1}
I20260621 07:56:59.366058 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.365613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35142)
0621 07:56:59.365872 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.365946 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":63,"threads_started":1}
W20260621 07:56:59.366245 19035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
W20260621 07:56:59.366623 18853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:59.367501 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.367266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35156)
0621 07:56:59.367402 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.367445 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260621 07:56:59.367587 19035 leader_election.cc:334] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:59.367702 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.367456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35162)
0621 07:56:59.367580 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.367622 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260621 07:56:59.367823 18853 leader_election.cc:334] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:59.369951 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.365924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:42685 (local address 127.0.0.1:49054)
0621 07:56:59.366401 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.366414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:59.366512 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:59.366808 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:59.366810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:59.366828 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:59.366940 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:59.366946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.368006 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.368014 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:59.368827 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:59.368836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.369517 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.369521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:59.369638 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:59.369670 (+ 32us) client_negotiation.cc:769] Sending connection context
0621 07:56:59.369748 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 07:56:59.369822 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":74,"threads_started":1}
I20260621 07:56:59.369968 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.366266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.11.223.188:42685)
0621 07:56:59.366544 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:59.366548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:59.366576 (+ 28us) server_negotiation.cc:407] Connection header received
0621 07:56:59.366617 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:59.366619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:59.366662 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:59.366748 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:59.367046 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.367820 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.368969 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.369416 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.369499 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:59.369535 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:59.369774 (+ 239us) server_negotiation.cc:299] Negotiation successful
0621 07:56:59.369847 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":75,"threads_started":1}
I20260621 07:56:59.370105 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.365934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43445 (local address 127.0.0.1:38552)
0621 07:56:59.366191 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.366205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:56:59.366300 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:56:59.366948 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:56:59.366951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:56:59.366970 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:56:59.367070 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:59.367077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.368179 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.368184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:59.369038 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:56:59.369047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.369690 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.369700 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:56:59.369801 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:59.369820 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 07:56:59.369896 (+ 76us) client_negotiation.cc:240] Negotiation successful
0621 07:56:59.369949 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":60,"threads_started":1}
I20260621 07:56:59.370213 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.366454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.11.223.189:43445)
0621 07:56:59.366686 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0621 07:56:59.366703 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 07:56:59.366715 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:56:59.366755 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:56:59.366758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:56:59.366801 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:56:59.366874 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:56:59.367212 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.368070 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.369147 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:56:59.369558 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:56:59.369694 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:56:59.369756 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0621 07:56:59.369918 (+ 162us) server_negotiation.cc:299] Negotiation successful
0621 07:56:59.369990 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":45,"threads_started":1}
I20260621 07:56:59.370633 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a30916872334f0aa08d515def96dc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3f33f8156bbc41968bef95bf0765d392" is_pre_election: true
I20260621 07:56:59.370764 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f33f8156bbc41968bef95bf0765d392" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5a30916872334f0aa08d515def96dc57" is_pre_election: true
I20260621 07:56:59.370858 18876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a30916872334f0aa08d515def96dc57 in term 1.
I20260621 07:56:59.370981 19059 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f33f8156bbc41968bef95bf0765d392 in term 1.
I20260621 07:56:59.371199 19035 leader_election.cc:302] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f33f8156bbc41968bef95bf0765d392, 5a30916872334f0aa08d515def96dc57; no voters: c46deeb03ea2484491608fb69140b413
I20260621 07:56:59.371327 18853 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f33f8156bbc41968bef95bf0765d392, 5a30916872334f0aa08d515def96dc57; no voters: c46deeb03ea2484491608fb69140b413
I20260621 07:56:59.371411 19573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260621 07:56:59.371464 19573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c46deeb03ea2484491608fb69140b413)
I20260621 07:56:59.371507 19573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:56:59.371640 19574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260621 07:56:59.371721 19574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c46deeb03ea2484491608fb69140b413)
I20260621 07:56:59.371771 19574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:56:59.372619 19573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:56:59.372776 19574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:56:59.372968 19573 leader_election.cc:288] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [CANDIDATE]: Term 2 election: Requested vote from peers c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495), 3f33f8156bbc41968bef95bf0765d392 (127.11.223.189:43445)
I20260621 07:56:59.373150 19574 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 2 election: Requested vote from peers c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495), 5a30916872334f0aa08d515def96dc57 (127.11.223.188:42685)
I20260621 07:56:59.373252 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.373056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35164)
0621 07:56:59.373134 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.373182 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260621 07:56:59.373548 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.373357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35174)
0621 07:56:59.373443 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.373494 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260621 07:56:59.373652 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f33f8156bbc41968bef95bf0765d392" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5a30916872334f0aa08d515def96dc57"
I20260621 07:56:59.373804 19059 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f33f8156bbc41968bef95bf0765d392 in current term 2: Already voted for candidate 5a30916872334f0aa08d515def96dc57 in this term.
I20260621 07:56:59.373873 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.373692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35176)
0621 07:56:59.373773 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.373825 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260621 07:56:59.373996 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a30916872334f0aa08d515def96dc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3f33f8156bbc41968bef95bf0765d392"
W20260621 07:56:59.374100 19035 leader_election.cc:334] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:59.374143 18876 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a30916872334f0aa08d515def96dc57 in current term 2: Already voted for candidate 3f33f8156bbc41968bef95bf0765d392 in this term.
I20260621 07:56:59.374508 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.374322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35184)
0621 07:56:59.374409 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.374448 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260621 07:56:59.374595 18853 leader_election.cc:334] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:56:59.374620 19035 leader_election.cc:302] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a30916872334f0aa08d515def96dc57; no voters: 3f33f8156bbc41968bef95bf0765d392, c46deeb03ea2484491608fb69140b413
I20260621 07:56:59.374668 18853 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f33f8156bbc41968bef95bf0765d392; no voters: 5a30916872334f0aa08d515def96dc57, c46deeb03ea2484491608fb69140b413
I20260621 07:56:59.374782 19573 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260621 07:56:59.374886 19574 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260621 07:56:59.857720 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.857267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:51213)
0621 07:56:59.857565 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.857622 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":112,"threads_started":1}
I20260621 07:56:59.857995 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.857628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:44425)
0621 07:56:59.857838 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.857900 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260621 07:56:59.859928 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:56:59.859563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:37447)
0621 07:56:59.859780 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0621 07:56:59.859839 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":62,"threads_started":1}
I20260621 07:57:00.858911 19586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:00.858432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:48905)
0621 07:57:00.858703 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:00.858784 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":67,"threads_started":1}
I20260621 07:57:00.859042 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:00.858577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:56241)
0621 07:57:00.858841 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:00.858905 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260621 07:57:00.861155 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:00.860575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:33775)
0621 07:57:00.860999 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:00.861071 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":73,"threads_started":1}
I20260621 07:57:01.297942 19589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:01.298070 19589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:57:01.298462 19589 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495), 5a30916872334f0aa08d515def96dc57 (127.11.223.188:42685)
I20260621 07:57:01.299031 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.298618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35426)
0621 07:57:01.298858 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.298929 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260621 07:57:01.299181 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f33f8156bbc41968bef95bf0765d392" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5a30916872334f0aa08d515def96dc57" is_pre_election: true
I20260621 07:57:01.299345 19059 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f33f8156bbc41968bef95bf0765d392 in term 2.
I20260621 07:57:01.299758 18853 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f33f8156bbc41968bef95bf0765d392, 5a30916872334f0aa08d515def96dc57; no voters:
I20260621 07:57:01.299847 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.299642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35432)
0621 07:57:01.299726 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.299771 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260621 07:57:01.299938 19589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260621 07:57:01.300000 18853 leader_election.cc:334] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:57:01.300020 19589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:01.300060 19589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 2 FOLLOWER]: Advancing to term 3
I20260621 07:57:01.301093 19589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:57:01.301403 19589 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 3 election: Requested vote from peers c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495), 5a30916872334f0aa08d515def96dc57 (127.11.223.188:42685)
I20260621 07:57:01.301692 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.301478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35438)
0621 07:57:01.301579 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.301629 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260621 07:57:01.301826 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f33f8156bbc41968bef95bf0765d392" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5a30916872334f0aa08d515def96dc57"
I20260621 07:57:01.301985 19059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 2 FOLLOWER]: Advancing to term 3
I20260621 07:57:01.302366 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.302198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35454)
0621 07:57:01.302286 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.302325 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260621 07:57:01.302464 18853 leader_election.cc:334] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
I20260621 07:57:01.303061 19059 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f33f8156bbc41968bef95bf0765d392 in term 3.
I20260621 07:57:01.303334 18853 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f33f8156bbc41968bef95bf0765d392, 5a30916872334f0aa08d515def96dc57; no voters: c46deeb03ea2484491608fb69140b413
I20260621 07:57:01.303473 19589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 3 FOLLOWER]: Leader election won for term 3
I20260621 07:57:01.303639 19589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [term 3 LEADER]: Becoming Leader. State: Replica: 3f33f8156bbc41968bef95bf0765d392, State: Running, Role: LEADER
I20260621 07:57:01.303895 19589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } }
I20260621 07:57:01.304903 19592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f33f8156bbc41968bef95bf0765d392. Latest consensus state: current_term: 3 leader_uuid: "3f33f8156bbc41968bef95bf0765d392" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:57:01.305027 19592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:01.305315 19594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:01.307397 19594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:01.307878 19594 catalog_manager.cc:1259] Loaded cluster ID: dfbc6cceb4724d06a63caf4c47b2f307
I20260621 07:57:01.308007 19594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:01.308594 19594 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:01.309038 19594 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Loaded TSK: 0
I20260621 07:57:01.309675 19594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:57:01.349785 18867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:57:01.352836 18867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:01.355604 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.355301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35474)
0621 07:57:01.355398 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.355515 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260621 07:57:01.355866 19059 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [term 3 FOLLOWER]: Refusing update from remote peer 3f33f8156bbc41968bef95bf0765d392: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260621 07:57:01.356398 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.356142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.0.0.1:35480)
0621 07:57:01.356255 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.356306 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 07:57:01.356436 19589 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260621 07:57:01.356590 18853 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 -> Peer c46deeb03ea2484491608fb69140b413 (127.11.223.190:45495): Couldn't send request to peer c46deeb03ea2484491608fb69140b413. Status: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260621 07:57:01.357810 19596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f33f8156bbc41968bef95bf0765d392. Latest consensus state: current_term: 3 leader_uuid: "3f33f8156bbc41968bef95bf0765d392" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:57:01.357932 19596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:57:01.358989 19592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3f33f8156bbc41968bef95bf0765d392" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:57:01.359138 19592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:01.359452 19596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3f33f8156bbc41968bef95bf0765d392" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c46deeb03ea2484491608fb69140b413" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45495 } } peers { permanent_uuid: "3f33f8156bbc41968bef95bf0765d392" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43445 } attrs { promote: false } } peers { permanent_uuid: "5a30916872334f0aa08d515def96dc57" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 42685 } attrs { promote: false } } }
I20260621 07:57:01.359570 19596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30916872334f0aa08d515def96dc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:57:01.368075 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.364429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45215 (local address 127.0.0.1:46910)
0621 07:57:01.364535 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.364549 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:01.364635 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:01.365061 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:01.365064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:01.365081 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:01.365186 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.365192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.366253 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.366255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.366926 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.366931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.367725 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.367728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.367805 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.367826 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:57:01.367909 (+ 83us) client_negotiation.cc:240] Negotiation successful
0621 07:57:01.367968 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260621 07:57:01.368538 19601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.364492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.11.223.131:45215)
0621 07:57:01.364787 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:01.364794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:01.364808 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:57:01.364878 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:01.364884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:01.364937 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:01.365296 (+ 359us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:01.365426 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.366168 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.367042 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.367620 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.367738 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.367782 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:01.368296 (+ 514us) server_negotiation.cc:299] Negotiation successful
0621 07:57:01.368404 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":74,"threads_started":1}
I20260621 07:57:01.368726 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.364950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:45175 (local address 127.0.0.1:54522)
0621 07:57:01.365167 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.365180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:01.365262 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:01.365719 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:01.365721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:01.365737 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:01.365812 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.365816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.366629 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.366634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.367525 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.367532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.368385 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.368389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.368470 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.368491 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:57:01.368540 (+ 49us) client_negotiation.cc:240] Negotiation successful
0621 07:57:01.368608 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":17,"thread_start_us":92,"threads_started":1}
I20260621 07:57:01.369045 19436 tablet_service.cc:1467] Processing CreateTablet for tablet 75abb93fb68b4d7a82f0b251508a5215 (DEFAULT_TABLE table=test-table [id=1b28aef4261448a6b214e5af28b9e7b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:01.369333 19436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75abb93fb68b4d7a82f0b251508a5215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:01.370298 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.365356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.11.223.130:42267)
0621 07:57:01.365578 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:01.365582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:01.366643 (+ 1061us) server_negotiation.cc:407] Connection header received
0621 07:57:01.366696 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:01.366698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:01.366743 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:01.366813 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:01.367164 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.367813 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.369288 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.369764 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.369888 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.369950 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:01.370116 (+ 166us) server_negotiation.cc:299] Negotiation successful
0621 07:57:01.370187 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260621 07:57:01.370293 19603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.365008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:42267 (local address 127.0.0.1:35252)
0621 07:57:01.365945 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.365972 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:01.366039 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:01.366906 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:01.366908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:01.366923 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:01.367049 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.367055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.368046 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.368049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.369187 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.369198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.369883 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.369887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.369988 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.370008 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 07:57:01.370102 (+ 94us) client_negotiation.cc:240] Negotiation successful
0621 07:57:01.370171 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":25,"thread_start_us":77,"threads_started":1}
I20260621 07:57:01.370831 19299 tablet_service.cc:1467] Processing CreateTablet for tablet 75abb93fb68b4d7a82f0b251508a5215 (DEFAULT_TABLE table=test-table [id=1b28aef4261448a6b214e5af28b9e7b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:01.371085 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75abb93fb68b4d7a82f0b251508a5215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:01.371729 19606 tablet_bootstrap.cc:492] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Bootstrap starting.
I20260621 07:57:01.372138 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.365278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54522 (local address 127.11.223.129:45175)
0621 07:57:01.365499 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:01.365503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:01.365516 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:57:01.365561 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:01.365563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:01.365604 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:01.365702 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:01.365926 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.366530 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.367631 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.368232 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.371874 (+ 3642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.371921 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:01.371953 (+ 32us) server_negotiation.cc:299] Negotiation successful
0621 07:57:01.372026 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260621 07:57:01.372632 19162 tablet_service.cc:1467] Processing CreateTablet for tablet 75abb93fb68b4d7a82f0b251508a5215 (DEFAULT_TABLE table=test-table [id=1b28aef4261448a6b214e5af28b9e7b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:01.372891 19162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75abb93fb68b4d7a82f0b251508a5215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:01.372893 19606 tablet_bootstrap.cc:654] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:01.374024 19606 tablet_bootstrap.cc:492] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: No bootstrap required, opened a new log
I20260621 07:57:01.374112 19606 ts_tablet_manager.cc:1397] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:01.374501 19606 raft_consensus.cc:348] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.374590 19606 raft_consensus.cc:374] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:01.374611 19606 raft_consensus.cc:729] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b442adc28634d16bf30af967fe1c4ea, State: Initialized, Role: FOLLOWER
I20260621 07:57:01.374742 19606 consensus_queue.cc:260] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.375010 19609 tablet_bootstrap.cc:492] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488: Bootstrap starting.
I20260621 07:57:01.375065 19606 ts_tablet_manager.cc:1428] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:01.373606 19608 tablet_bootstrap.cc:492] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Bootstrap starting.
I20260621 07:57:01.375820 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.375449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:37963)
0621 07:57:01.375679 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.375733 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260621 07:57:01.376192 19609 tablet_bootstrap.cc:654] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:01.376468 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.376213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:35693)
0621 07:57:01.376358 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.376403 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260621 07:57:01.376557 19608 tablet_bootstrap.cc:654] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:01.377656 19609 tablet_bootstrap.cc:492] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488: No bootstrap required, opened a new log
I20260621 07:57:01.377750 19609 ts_tablet_manager.cc:1397] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260621 07:57:01.378192 19609 raft_consensus.cc:348] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.378326 19609 raft_consensus.cc:374] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:01.378324 19504 heartbeater.cc:503] Master 127.11.223.189:43445 was elected leader, sending a full tablet report...
I20260621 07:57:01.378364 19609 raft_consensus.cc:729] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668417c1c7a7417fbfd08bc456a59488, State: Initialized, Role: FOLLOWER
I20260621 07:57:01.378485 19609 consensus_queue.cc:260] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.378798 19609 ts_tablet_manager.cc:1428] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:01.379695 19608 tablet_bootstrap.cc:492] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: No bootstrap required, opened a new log
I20260621 07:57:01.379781 19608 ts_tablet_manager.cc:1397] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260621 07:57:01.380129 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.379263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:59061)
0621 07:57:01.379994 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.380047 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":83,"threads_started":1}
I20260621 07:57:01.380242 19608 raft_consensus.cc:348] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.380389 19608 raft_consensus.cc:374] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:01.380421 19608 raft_consensus.cc:729] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88efbf17dec149638cba8a3b8fc78145, State: Initialized, Role: FOLLOWER
I20260621 07:57:01.380548 19608 consensus_queue.cc:260] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.380879 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.380648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:46401)
0621 07:57:01.380769 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.380822 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 07:57:01.380903 19230 heartbeater.cc:503] Master 127.11.223.189:43445 was elected leader, sending a full tablet report...
I20260621 07:57:01.381513 19608 ts_tablet_manager.cc:1428] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:01.382220 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.381162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:44367)
0621 07:57:01.381908 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.381979 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":91,"threads_started":1}
I20260621 07:57:01.382874 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.382622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:60463)
0621 07:57:01.382742 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.382813 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260621 07:57:01.382938 19367 heartbeater.cc:503] Master 127.11.223.189:43445 was elected leader, sending a full tablet report...
I20260621 07:57:01.604159 19616 raft_consensus.cc:482] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:01.604286 19616 raft_consensus.cc:504] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.605412 19616 leader_election.cc:288] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215), 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175)
I20260621 07:57:01.609499 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.605764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:47771 (local address 127.11.223.129:45175)
0621 07:57:01.605926 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:01.605930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:01.606176 (+ 246us) server_negotiation.cc:407] Connection header received
0621 07:57:01.606338 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:01.606341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:01.606390 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:01.606476 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:01.606744 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.607235 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.608442 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.608955 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.609085 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.609137 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:01.609321 (+ 184us) server_negotiation.cc:299] Negotiation successful
0621 07:57:01.609385 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:57:01.609532 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.605635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:45175 (local address 127.11.223.130:47771)
0621 07:57:01.606078 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.606090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:01.606219 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:01.606487 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:01.606490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:01.606505 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:01.606626 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.606631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.607351 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.607355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.608278 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.608287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.609085 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.609089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.609191 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.609212 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:57:01.609307 (+ 95us) client_negotiation.cc:240] Negotiation successful
0621 07:57:01.609371 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":126,"thread_start_us":140,"threads_started":1}
I20260621 07:57:01.609642 19601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.605713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:33989 (local address 127.11.223.131:45215)
0621 07:57:01.605860 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:01.605865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:01.606003 (+ 138us) server_negotiation.cc:407] Connection header received
0621 07:57:01.606175 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:01.606178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:01.606216 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:01.606295 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:01.606604 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.607245 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.608613 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.609050 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.609178 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.609231 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:01.609434 (+ 203us) server_negotiation.cc:299] Negotiation successful
0621 07:57:01.609503 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:57:01.609603 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.605620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45215 (local address 127.11.223.130:33989)
0621 07:57:01.605914 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.605927 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:01.606050 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:01.606317 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:01.606320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:01.606337 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:01.606455 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.606460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.607354 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.607359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.608472 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:01.608481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.609164 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.609167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:01.609268 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.609288 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 07:57:01.609379 (+ 91us) client_negotiation.cc:240] Negotiation successful
0621 07:57:01.609452 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":57}
I20260621 07:57:01.610119 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75abb93fb68b4d7a82f0b251508a5215" candidate_uuid: "88efbf17dec149638cba8a3b8fc78145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b442adc28634d16bf30af967fe1c4ea" is_pre_election: true
I20260621 07:57:01.610263 19456 raft_consensus.cc:2454] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88efbf17dec149638cba8a3b8fc78145 in term 0.
I20260621 07:57:01.610540 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75abb93fb68b4d7a82f0b251508a5215" candidate_uuid: "88efbf17dec149638cba8a3b8fc78145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668417c1c7a7417fbfd08bc456a59488" is_pre_election: true
I20260621 07:57:01.610592 19252 leader_election.cc:302] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88efbf17dec149638cba8a3b8fc78145, 9b442adc28634d16bf30af967fe1c4ea; no voters:
I20260621 07:57:01.610770 19182 raft_consensus.cc:2454] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88efbf17dec149638cba8a3b8fc78145 in term 0.
I20260621 07:57:01.610811 19616 raft_consensus.cc:2781] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:01.610858 19616 raft_consensus.cc:482] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:01.610903 19616 raft_consensus.cc:3037] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:01.612880 19616 raft_consensus.cc:504] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.613292 19616 leader_election.cc:288] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [CANDIDATE]: Term 1 election: Requested vote from peers 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215), 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175)
I20260621 07:57:01.613518 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75abb93fb68b4d7a82f0b251508a5215" candidate_uuid: "88efbf17dec149638cba8a3b8fc78145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b442adc28634d16bf30af967fe1c4ea"
I20260621 07:57:01.613636 19456 raft_consensus.cc:3037] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:01.613655 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75abb93fb68b4d7a82f0b251508a5215" candidate_uuid: "88efbf17dec149638cba8a3b8fc78145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668417c1c7a7417fbfd08bc456a59488"
I20260621 07:57:01.613768 19182 raft_consensus.cc:3037] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:01.614500 19456 raft_consensus.cc:2454] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88efbf17dec149638cba8a3b8fc78145 in term 1.
I20260621 07:57:01.614733 19182 raft_consensus.cc:2454] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88efbf17dec149638cba8a3b8fc78145 in term 1.
I20260621 07:57:01.614743 19252 leader_election.cc:302] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88efbf17dec149638cba8a3b8fc78145, 9b442adc28634d16bf30af967fe1c4ea; no voters:
I20260621 07:57:01.614902 19616 raft_consensus.cc:2781] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:01.615096 19616 raft_consensus.cc:686] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 1 LEADER]: Becoming Leader. State: Replica: 88efbf17dec149638cba8a3b8fc78145, State: Running, Role: LEADER
I20260621 07:57:01.615322 19616 consensus_queue.cc:237] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } }
I20260621 07:57:01.615792 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.615539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:38925)
0621 07:57:01.615681 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.615733 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260621 07:57:01.617594 18867 catalog_manager.cc:5495] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "88efbf17dec149638cba8a3b8fc78145" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88efbf17dec149638cba8a3b8fc78145" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 } health_report { overall_health: UNKNOWN } } }
I20260621 07:57:01.660698 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.657118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35256 (local address 127.11.223.130:42267)
0621 07:57:01.657255 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:01.657259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:01.657272 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:57:01.657414 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:01.657417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:01.657481 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:01.657585 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:01.657837 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.658325 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.659227 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:01.659430 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:01.659542 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:01.659810 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:01.660285 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:01.660436 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:01.660506 (+ 70us) server_negotiation.cc:299] Negotiation successful
0621 07:57:01.660557 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 07:57:01.663129 19456 raft_consensus.cc:1261] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Refusing update from remote peer 88efbf17dec149638cba8a3b8fc78145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:01.663365 19182 raft_consensus.cc:1261] T 75abb93fb68b4d7a82f0b251508a5215 P 668417c1c7a7417fbfd08bc456a59488 [term 1 FOLLOWER]: Refusing update from remote peer 88efbf17dec149638cba8a3b8fc78145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:01.663637 19619 consensus_queue.cc:1035] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b442adc28634d16bf30af967fe1c4ea" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:01.663888 19616 consensus_queue.cc:1035] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668417c1c7a7417fbfd08bc456a59488" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:01.665808 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.665570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:36297)
0621 07:57:01.665692 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.665745 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260621 07:57:01.666180 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.665917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:36507)
0621 07:57:01.666061 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.666112 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260621 07:57:01.668692 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.668439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:41899)
0621 07:57:01.668581 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.668628 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260621 07:57:01.669488 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.669292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.130:38187)
0621 07:57:01.669376 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.669424 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260621 07:57:01.669900 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.669510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.129:34049)
0621 07:57:01.669794 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.669841 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
I20260621 07:57:01.670989 19622 mvcc.cc:205] Tried to move back new op lower bound from 7299189234327707648 to 7299189234139049984. Current Snapshot: MvccSnapshot[applied={T|T < 7299189234327707648}]
I20260621 07:57:01.671541 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.671325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45495 (local address 127.11.223.131:47943)
0621 07:57:01.671453 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.671494 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260621 07:57:01.677848 18867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:01.677994 18867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:01.681730 18867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f33f8156bbc41968bef95bf0765d392: Sending DeleteTablet for 3 replicas of tablet 75abb93fb68b4d7a82f0b251508a5215
I20260621 07:57:01.682225 19436 tablet_service.cc:1514] Processing DeleteTablet for tablet 75abb93fb68b4d7a82f0b251508a5215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:01 UTC) from {username='oryx', principal='oryx/127.11.223.189@KRBTEST.COM'} at 127.0.0.1:46910
I20260621 07:57:01.682614 19299 tablet_service.cc:1514] Processing DeleteTablet for tablet 75abb93fb68b4d7a82f0b251508a5215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:01 UTC) from {username='oryx', principal='oryx/127.11.223.189@KRBTEST.COM'} at 127.0.0.1:35252
I20260621 07:57:01.682859 19162 tablet_service.cc:1514] Processing DeleteTablet for tablet 75abb93fb68b4d7a82f0b251508a5215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:01 UTC) from {username='oryx', principal='oryx/127.11.223.189@KRBTEST.COM'} at 127.0.0.1:54522
I20260621 07:57:01.683049 19634 tablet_replica.cc:330] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: stopping tablet replica
I20260621 07:57:01.683202 19634 raft_consensus.cc:2229] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:57:01.683362 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 19011
I20260621 07:57:01.683591 19634 raft_consensus.cc:2258] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:01.684495 19634 ts_tablet_manager.cc:1905] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:01.685235 19633 tablet_replica.cc:330] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: stopping tablet replica
I20260621 07:57:01.685354 19633 raft_consensus.cc:2229] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:01.685468 19633 raft_consensus.cc:2258] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:01.686093 19633 ts_tablet_manager.cc:1905] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:01.687896 19634 ts_tablet_manager.cc:1918] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:01.688021 19634 log.cc:1196] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/75abb93fb68b4d7a82f0b251508a5215
I20260621 07:57:01.688419 19634 ts_tablet_manager.cc:1939] T 75abb93fb68b4d7a82f0b251508a5215 P 88efbf17dec149638cba8a3b8fc78145: Deleting consensus metadata
I20260621 07:57:01.688609 19633 ts_tablet_manager.cc:1918] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:01.688684 19633 log.cc:1196] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/75abb93fb68b4d7a82f0b251508a5215
I20260621 07:57:01.688958 19633 ts_tablet_manager.cc:1939] T 75abb93fb68b4d7a82f0b251508a5215 P 9b442adc28634d16bf30af967fe1c4ea: Deleting consensus metadata
I20260621 07:57:01.689317 18854 catalog_manager.cc:4841] TS 88efbf17dec149638cba8a3b8fc78145 (127.11.223.130:42267): tablet 75abb93fb68b4d7a82f0b251508a5215 (table test-table [id=1b28aef4261448a6b214e5af28b9e7b0]) successfully deleted
I20260621 07:57:01.689698 18851 catalog_manager.cc:4841] TS 9b442adc28634d16bf30af967fe1c4ea (127.11.223.131:45215): tablet 75abb93fb68b4d7a82f0b251508a5215 (table test-table [id=1b28aef4261448a6b214e5af28b9e7b0]) successfully deleted
I20260621 07:57:01.692884 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 19236
I20260621 07:57:01.693099 19603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:01.692818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:45175 (local address 127.0.0.1:54534)
0621 07:57:01.692944 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:01.693007 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260621 07:57:01.693276 18853 catalog_manager.cc:4584] TS 668417c1c7a7417fbfd08bc456a59488 (127.11.223.129:45175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75abb93fb68b4d7a82f0b251508a5215: Network error: Client connection negotiation failed: client connection to 127.11.223.129:45175: connect: Connection refused (error 111)
I20260621 07:57:01.697867 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 19372
I20260621 07:57:01.703311 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18730
I20260621 07:57:01.708875 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 18839
2026-06-21T07:57:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7408 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:57:01 dist-test-slave-0pn3 krb5kdc[19638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:57:01 dist-test-slave-0pn3 krb5kdc[19638](info): set up 2 sockets
Jun 21 07:57:01 dist-test-slave-0pn3 krb5kdc[19638](info): commencing operation
krb5kdc: starting...
W20260621 07:57:02.705142 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:57:02 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:57:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab.
Entry for principal oryx/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab.
I20260621 07:57:02.814975 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46137
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab
--principal=oryx/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:46137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:02.916095 19651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:02.916358 19651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:02.916435 19651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:02.920195 19651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:02.920274 19651 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260621 07:57:02.920305 19651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:02.920369 19651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:02.920419 19651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:02.920459 19651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:02.925325 19651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:46137
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.190.keytab
--principal=oryx/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:02.926201 19651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:02.927075 19651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:02.934710 19659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:02.934789 19651 server_base.cc:1034] running on GCE node
W20260621 07:57:02.934729 19656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:02.934706 19657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:02.935457 19651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:02.936450 19651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:02.937623 19651 hybrid_clock.cc:648] HybridClock initialized: now 1782028622937608 us; error 31 us; skew 500 ppm
Jun 21 07:57:02 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028622, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:02.940234 19651 init.cc:377] Logged in from keytab as oryx/127.11.223.190@KRBTEST.COM (short username oryx)
I20260621 07:57:02.941028 19651 webserver.cc:460] Webserver started at http://127.11.223.190:36913/ using document root <none> and password file <none>
I20260621 07:57:02.941553 19651 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:02.941641 19651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:02.941851 19651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:02.943652 19651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "a72476c7ec5e4918b60ed604829bad33"
format_stamp: "Formatted at 2026-06-21 07:57:02 on dist-test-slave-0pn3"
server_key: "1c9adcb924c9236007128de65417a09f"
server_key_iv: "467eb6d9d7ee37101688f7b21bf23a68"
server_key_version: "encryptionkey@0"
I20260621 07:57:02.944176 19651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "a72476c7ec5e4918b60ed604829bad33"
format_stamp: "Formatted at 2026-06-21 07:57:02 on dist-test-slave-0pn3"
server_key: "1c9adcb924c9236007128de65417a09f"
server_key_iv: "467eb6d9d7ee37101688f7b21bf23a68"
server_key_version: "encryptionkey@0"
I20260621 07:57:02.947580 19651 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:57:02.949806 19666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:02.950906 19651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:02.951042 19651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "a72476c7ec5e4918b60ed604829bad33"
format_stamp: "Formatted at 2026-06-21 07:57:02 on dist-test-slave-0pn3"
server_key: "1c9adcb924c9236007128de65417a09f"
server_key_iv: "467eb6d9d7ee37101688f7b21bf23a68"
server_key_version: "encryptionkey@0"
I20260621 07:57:02.951159 19651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:02.966270 19651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:02.969209 19651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:02.969394 19651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:02.976454 19651 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:46137
I20260621 07:57:02.976477 19728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:46137 every 8 connection(s)
I20260621 07:57:02.977226 19651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:57:02.979629 19729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:02.980300 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 19651
I20260621 07:57:02.980484 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:57:02.980767 12158 external_mini_cluster.cc:1428] Setting key 36b0f6930ee3094a2d38a7cc7e3d8ab5
I20260621 07:57:02.984910 19729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Bootstrap starting.
I20260621 07:57:02.986843 19729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:02.987617 19729 log.cc:824] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Log is configured to *not* fsync() on all Append() calls
Jun 21 07:57:02 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:02.989625 19729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: No bootstrap required, opened a new log
I20260621 07:57:02.992293 19729 raft_consensus.cc:348] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72476c7ec5e4918b60ed604829bad33" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46137 } }
I20260621 07:57:02.992606 19729 raft_consensus.cc:374] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:02.992697 19729 raft_consensus.cc:729] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a72476c7ec5e4918b60ed604829bad33, State: Initialized, Role: FOLLOWER
I20260621 07:57:02.993126 19729 consensus_queue.cc:260] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72476c7ec5e4918b60ed604829bad33" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46137 } }
I20260621 07:57:02.993269 19729 raft_consensus.cc:388] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:57:02.993330 19729 raft_consensus.cc:482] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:57:02.993431 19729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:02.994228 19729 raft_consensus.cc:504] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72476c7ec5e4918b60ed604829bad33" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46137 } }
I20260621 07:57:02.994302 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:02.982001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.11.223.190:46137)
0621 07:57:02.982387 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:02.982399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:02.982430 (+ 31us) server_negotiation.cc:407] Connection header received
0621 07:57:02.983030 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:02.983044 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:02.983337 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:02.983642 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:02.984383 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:02.985300 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:02.986128 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:02.986566 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:02.986745 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:02.989223 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:02.989229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:02.989243 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:02.989274 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:02.991472 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:02.991920 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:02.991925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:02.991929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:02.992027 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:02.992380 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:02.992383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:02.992385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:02.992887 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:02.993097 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:02.993354 (+ 257us) server_negotiation.cc:299] Negotiation successful
0621 07:57:02.993645 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":123,"threads_started":1}
I20260621 07:57:02.994592 19729 leader_election.cc:302] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a72476c7ec5e4918b60ed604829bad33; no voters:
I20260621 07:57:02.994897 19729 leader_election.cc:288] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:57:02.994975 19734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:02.995214 19734 raft_consensus.cc:686] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [term 1 LEADER]: Becoming Leader. State: Replica: a72476c7ec5e4918b60ed604829bad33, State: Running, Role: LEADER
I20260621 07:57:02.995486 19734 consensus_queue.cc:237] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72476c7ec5e4918b60ed604829bad33" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46137 } }
I20260621 07:57:02.995920 19729 sys_catalog.cc:564] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:57:02.996994 19735 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a72476c7ec5e4918b60ed604829bad33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72476c7ec5e4918b60ed604829bad33" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46137 } } }
I20260621 07:57:02.997152 19735 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:02.997447 19736 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a72476c7ec5e4918b60ed604829bad33. Latest consensus state: current_term: 1 leader_uuid: "a72476c7ec5e4918b60ed604829bad33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72476c7ec5e4918b60ed604829bad33" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46137 } } }
I20260621 07:57:02.997551 19736 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:02.997504 19743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:02.999934 19743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:03.005264 19743 catalog_manager.cc:1347] Generated new cluster ID: 770ef5913f734ecd926d29450576fcb5
I20260621 07:57:03.005322 19743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:03.019369 19743 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:57:03.020495 19743 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:03.026582 19743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Generated new TSK 0
I20260621 07:57:03.027243 19743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab.
Entry for principal oryx/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab.
I20260621 07:57:03.056089 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46137
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:03.164013 19755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:03.164252 19755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:03.164295 19755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:03.168124 19755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:03.168172 19755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:03.168210 19755 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:57:03.168239 19755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:57:03.173308 19755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:03.174226 19755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:03.175156 19755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:57:03.184137 19755 server_base.cc:1034] running on GCE node
W20260621 07:57:03.184146 19760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:03.184149 19761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:03.184280 19763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:03.184640 19755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:03.185139 19755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:03.186321 19755 hybrid_clock.cc:648] HybridClock initialized: now 1782028623186308 us; error 33 us; skew 500 ppm
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:03.188985 19755 init.cc:377] Logged in from keytab as oryx/127.11.223.129@KRBTEST.COM (short username oryx)
I20260621 07:57:03.189764 19755 webserver.cc:460] Webserver started at http://127.11.223.129:41959/ using document root <none> and password file <none>
I20260621 07:57:03.190254 19755 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:03.190308 19755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:03.190459 19755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:03.192308 19755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "6670959e07544418990c6ab6b835ba8c"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "74789d3a2a9dd512a639c6279156a8c4"
server_key_iv: "b2d49b60adb7d5e72c18c3dabd1b5e4b"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.192881 19755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "6670959e07544418990c6ab6b835ba8c"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "74789d3a2a9dd512a639c6279156a8c4"
server_key_iv: "b2d49b60adb7d5e72c18c3dabd1b5e4b"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.196261 19755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:57:03.198616 19770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:03.199714 19755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:03.199826 19755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "6670959e07544418990c6ab6b835ba8c"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "74789d3a2a9dd512a639c6279156a8c4"
server_key_iv: "b2d49b60adb7d5e72c18c3dabd1b5e4b"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.199918 19755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:03.215767 19755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:03.218400 19755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:03.218590 19755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:03.219890 19755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:03.219964 19755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:03.220041 19755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:03.220086 19755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:03.230532 19755 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:36911
I20260621 07:57:03.230604 19885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:36911 every 8 connection(s)
I20260621 07:57:03.231483 19755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:57:03.234157 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.220753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:57111 (local address 127.11.223.190:46137)
0621 07:57:03.220876 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.220880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.221835 (+ 955us) server_negotiation.cc:407] Connection header received
0621 07:57:03.222792 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.222796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.222854 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.222914 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.224667 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.225573 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.227354 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.227770 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.227877 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.230728 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.230733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.230736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.230762 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.232492 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.232928 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.232930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.232931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.233002 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.233290 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.233292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.233293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.233447 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.233520 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.233935 (+ 415us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.234039 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:57:03.234894 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.221246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.11.223.129:57111)
0621 07:57:03.221669 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.221715 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.222545 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.223340 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.223366 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.223873 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:03.224494 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.224514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.225724 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.225730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.227201 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.227211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.227907 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.227911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.228094 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.228717 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:03.228742 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:03.230517 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:03.232584 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.232588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.232598 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.232841 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.233092 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.233094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.233095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.233214 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.233530 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:03.233535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:03.233739 (+ 204us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.233885 (+ 146us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.234097 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":127,"threads_started":1}
I20260621 07:57:03.236020 19886 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46137
I20260621 07:57:03.236213 19886 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:03.236744 19886 heartbeater.cc:511] Master 127.11.223.190:46137 requested a full tablet report, sending...
I20260621 07:57:03.238247 19683 ts_manager.cc:194] Registered new tserver with Master: 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911)
I20260621 07:57:03.239311 19683 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:57111
I20260621 07:57:03.241459 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 19755
I20260621 07:57:03.241623 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:57:03.241958 12158 external_mini_cluster.cc:1428] Setting key 5e52b71000b7ff388c13ec0dbb7c82ee
I20260621 07:57:03.246963 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.238210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.11.223.129:40621)
0621 07:57:03.239916 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.239930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.240031 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.240273 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.240275 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.240483 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:03.240576 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.240582 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.241577 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.241581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.242382 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.242391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.242956 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.242961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.243061 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.243549 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:03.243561 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:03.243853 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:03.245583 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.245589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.245592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.245785 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.246073 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.246075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.246077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.246161 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.246587 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:03.246589 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:03.246657 (+ 68us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.246735 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.246830 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1631,"thread_start_us":90,"threads_started":1}
I20260621 07:57:03.247169 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.238275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:40621 (local address 127.11.223.190:46137)
0621 07:57:03.238380 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.238383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.240004 (+ 1621us) server_negotiation.cc:407] Connection header received
0621 07:57:03.240124 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.240127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.240170 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.240252 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.240689 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.241421 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.242512 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.242818 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.242911 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.243990 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.243997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.244003 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.244032 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.245470 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.245884 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.245890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.245895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.245980 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.246257 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.246264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.246268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.246495 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.246598 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.246814 (+ 216us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.246940 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for oryx/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab.
Entry for principal oryx/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab.
I20260621 07:57:03.266794 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46137
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:03.370373 19895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:03.370599 19895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:03.370640 19895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:03.374490 19895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:03.374534 19895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:03.374568 19895 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:57:03.374596 19895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:57:03.379225 19895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:03.380059 19895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:03.381057 19895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:03.389752 19901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:03.389703 19903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:03.389760 19900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:03.389922 19895 server_base.cc:1034] running on GCE node
I20260621 07:57:03.390282 19895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:03.390889 19895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:03.392058 19895 hybrid_clock.cc:648] HybridClock initialized: now 1782028623392032 us; error 44 us; skew 500 ppm
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:03.394660 19895 init.cc:377] Logged in from keytab as oryx/127.11.223.130@KRBTEST.COM (short username oryx)
I20260621 07:57:03.395407 19895 webserver.cc:460] Webserver started at http://127.11.223.130:40045/ using document root <none> and password file <none>
I20260621 07:57:03.395931 19895 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:03.396013 19895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:03.396237 19895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:03.398209 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "12ed09a4ecea46448a2468c7b95c375b"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "9b6fb2ce33cef67fd56abeaf542326f3"
server_key_iv: "2a1bb86073d50f4ce2e3d81c6d837117"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.398723 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "12ed09a4ecea46448a2468c7b95c375b"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "9b6fb2ce33cef67fd56abeaf542326f3"
server_key_iv: "2a1bb86073d50f4ce2e3d81c6d837117"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.402062 19895 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:57:03.404186 19910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:03.405256 19895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:03.405380 19895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "12ed09a4ecea46448a2468c7b95c375b"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "9b6fb2ce33cef67fd56abeaf542326f3"
server_key_iv: "2a1bb86073d50f4ce2e3d81c6d837117"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.405475 19895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:03.419073 19895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:03.422055 19895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:03.422238 19895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:03.423524 19895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:03.423590 19895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:03.423666 19895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:03.423702 19895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:03.434162 19895 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:33855
I20260621 07:57:03.434434 20025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:33855 every 8 connection(s)
I20260621 07:57:03.435210 19895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:57:03.437361 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.424218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:44761 (local address 127.11.223.190:46137)
0621 07:57:03.424383 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.424388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.425241 (+ 853us) server_negotiation.cc:407] Connection header received
0621 07:57:03.426474 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.426479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.426545 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.426638 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.428133 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.428754 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.430620 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.430935 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.431054 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.433805 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.433808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.433810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.433833 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.435436 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.435997 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.436000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.436003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.436079 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.436478 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.436481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.436483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.436660 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.436737 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.437140 (+ 403us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.437229 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 07:57:03.438035 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.424618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.11.223.130:44761)
0621 07:57:03.425064 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.425127 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.426177 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.426843 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.426855 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.427364 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:03.427991 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.428010 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.428904 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.428908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.430474 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.430485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.431096 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.431100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.431271 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.432214 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:03.432229 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:03.433641 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:03.435560 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.435566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.435580 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.435878 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.436203 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.436205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.436206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.436382 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.436755 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:03.436760 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:03.436945 (+ 185us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.437079 (+ 134us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.437319 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260621 07:57:03.439201 20026 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46137
I20260621 07:57:03.439350 20026 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:03.439777 20026 heartbeater.cc:511] Master 127.11.223.190:46137 requested a full tablet report, sending...
I20260621 07:57:03.440728 19682 ts_manager.cc:194] Registered new tserver with Master: 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855)
I20260621 07:57:03.441375 19682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:44761
I20260621 07:57:03.442119 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 19895
I20260621 07:57:03.442256 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:57:03.442515 12158 external_mini_cluster.cc:1428] Setting key b14598e419e4dc55ff4094857e090cd9
I20260621 07:57:03.448419 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.441185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:55077 (local address 127.11.223.190:46137)
0621 07:57:03.441316 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.441320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.441528 (+ 208us) server_negotiation.cc:407] Connection header received
0621 07:57:03.441653 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.441656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.441698 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.441774 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.442285 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.442995 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.444016 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.444210 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.444283 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.445452 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.445455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.445456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.445476 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.446820 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.447276 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.447278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.447279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.447344 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.447636 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.447638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.447639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.447805 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.447910 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.448160 (+ 250us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.448254 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:57:03.448431 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.441107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.11.223.130:55077)
0621 07:57:03.441460 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.441475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.441562 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.441789 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.441791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.442013 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:03.442152 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.442160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.443140 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.443143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.443906 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.443916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.444358 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.444366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.444489 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.444895 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:03.444914 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:03.445320 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:03.446926 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.446932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.446937 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.447183 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.447434 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.447440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.447444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.447543 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.447912 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:03.447915 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:03.448004 (+ 89us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.448109 (+ 105us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.448243 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for oryx/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab.
Entry for principal oryx/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab.
I20260621 07:57:03.467043 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46137
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:03.570883 20035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:03.571130 20035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:03.571171 20035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:03.574903 20035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:03.574949 20035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:03.574985 20035 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260621 07:57:03.575013 20035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:57:03.579512 20035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.223.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:03.580368 20035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:03.581274 20035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:03.589768 20041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:03.589799 20040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:03.589735 20043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:03.589893 20035 server_base.cc:1034] running on GCE node
I20260621 07:57:03.590471 20035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:03.591042 20035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:03.592232 20035 hybrid_clock.cc:648] HybridClock initialized: now 1782028623592218 us; error 46 us; skew 500 ppm
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:03.594695 20035 init.cc:377] Logged in from keytab as oryx/127.11.223.131@KRBTEST.COM (short username oryx)
I20260621 07:57:03.595489 20035 webserver.cc:460] Webserver started at http://127.11.223.131:45287/ using document root <none> and password file <none>
I20260621 07:57:03.596056 20035 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:03.596149 20035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:03.596402 20035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:03.598484 20035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "6e79c78bee5d494bbdb9925fc808203b"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "97a16ba337d8db7bce86a932ac7816b6"
server_key_iv: "9118f123d1be7a56c6e09acec08ae533"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.599004 20035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "6e79c78bee5d494bbdb9925fc808203b"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "97a16ba337d8db7bce86a932ac7816b6"
server_key_iv: "9118f123d1be7a56c6e09acec08ae533"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.602398 20035 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:57:03.604755 20050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:03.605852 20035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:57:03.606017 20035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "6e79c78bee5d494bbdb9925fc808203b"
format_stamp: "Formatted at 2026-06-21 07:57:03 on dist-test-slave-0pn3"
server_key: "97a16ba337d8db7bce86a932ac7816b6"
server_key_iv: "9118f123d1be7a56c6e09acec08ae533"
server_key_version: "encryptionkey@0"
I20260621 07:57:03.606132 20035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:03.619738 20035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:03.622258 20035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:03.622437 20035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:03.623720 20035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:03.623790 20035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:03.623853 20035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:03.623907 20035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:03.634816 20035 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:38277
I20260621 07:57:03.634877 20165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:38277 every 8 connection(s)
I20260621 07:57:03.635744 20035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:57:03.637058 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.624365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:38171 (local address 127.11.223.190:46137)
0621 07:57:03.624496 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.624502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.625311 (+ 809us) server_negotiation.cc:407] Connection header received
0621 07:57:03.626499 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.626502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.626555 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.626657 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.627721 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.628397 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.629682 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.629912 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.630020 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.633295 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.633298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.633301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.633332 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.634962 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.635426 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.635432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.635437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.635525 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.635918 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.635921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.635923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.636157 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.636252 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.636797 (+ 545us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.636888 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:57:03.637738 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.624705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.11.223.131:38171)
0621 07:57:03.625140 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.625196 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.626277 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.626815 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.626826 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.627189 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:03.627586 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.627597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.628509 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.628512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.629558 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.629568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.630054 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.630058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.630255 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.630833 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:03.630850 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:03.633142 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:03.635066 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.635071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.635080 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.635323 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.635620 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.635624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.635626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.635756 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.636303 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:03.636339 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:03.636605 (+ 266us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.636756 (+ 151us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.636959 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":100,"threads_started":1}
I20260621 07:57:03.638845 20166 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46137
I20260621 07:57:03.638983 20166 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:03.639570 20166 heartbeater.cc:511] Master 127.11.223.190:46137 requested a full tablet report, sending...
I20260621 07:57:03.640524 19683 ts_manager.cc:194] Registered new tserver with Master: 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:03.641279 19683 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:38171
I20260621 07:57:03.642086 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20035
I20260621 07:57:03.642218 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:57:03.642481 12158 external_mini_cluster.cc:1428] Setting key bd8b41891df2f151e4ac831886523c9c
I20260621 07:57:03.644256 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:57:03.647648 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.640850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:58481 (local address 127.11.223.190:46137)
0621 07:57:03.640966 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.640970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.641121 (+ 151us) server_negotiation.cc:407] Connection header received
0621 07:57:03.641196 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.641199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.641239 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.641336 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.641747 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.642481 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.643514 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.643737 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.643842 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.644813 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.644816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.644819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.644840 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.646217 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.646607 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.646612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.646616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.646688 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.646945 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.646949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.646950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.647104 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.647199 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.647433 (+ 234us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.647520 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:57:03.647660 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.640754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.11.223.131:58481)
0621 07:57:03.641048 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.641059 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.641137 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.641338 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.641339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.641531 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:03.641657 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.641663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.642640 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.642643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.643398 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.643407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.643844 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.643847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.643933 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.644383 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:03.644395 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:03.644676 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:03.646336 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.646339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.646341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.646516 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.646771 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:03.646775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:03.646777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:03.646862 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:03.647191 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:03.647194 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:03.647273 (+ 79us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.647374 (+ 101us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.647488 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":80,"threads_started":1}
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
W20260621 07:57:03.666985 19732 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:03.661480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.11.223.190:46137)
0621 07:57:03.661613 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.661617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.661628 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:03.661658 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.661661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.661692 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.661754 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.662383 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.662980 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.663765 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.663974 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.664045 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.666891 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38738: BlockingRecv error: recv got EOF from 127.0.0.1:38738 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 21 07:57:03 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:03.675568 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.668298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.11.223.190:46137)
0621 07:57:03.668432 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.668436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.668556 (+ 120us) server_negotiation.cc:407] Connection header received
0621 07:57:03.668705 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.668708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.668750 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.668824 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:03.669351 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.669849 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.670604 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.670826 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.670931 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.672945 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:03.672948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:03.672951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:03.672974 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:03.674043 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.674461 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.674466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.674468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.674546 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:03.674808 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:03.674812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:03.674813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:03.674972 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:03.675109 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.675320 (+ 211us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.675424 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 07:57:03.678061 19683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:57:03.680999 19683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:03.694048 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.688467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38277 (local address 127.0.0.1:59736)
0621 07:57:03.688764 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.688812 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.688930 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.690285 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.690288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.690303 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:03.690405 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.690411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.691393 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.691399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.692725 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.692735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.693546 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.693549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.693662 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.693696 (+ 34us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.693799 (+ 103us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.693904 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260621 07:57:03.694232 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.688543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36911 (local address 127.0.0.1:43896)
0621 07:57:03.689299 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.689311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.689401 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.690131 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.690137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.690159 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:03.690285 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.690292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.691675 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.691677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.692787 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.692798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.693797 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.693800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.693885 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.693909 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.693965 (+ 56us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.694014 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260621 07:57:03.694456 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.689129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33855 (local address 127.0.0.1:57752)
0621 07:57:03.689635 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.689648 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.689726 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.690446 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.690449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.690488 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:03.690583 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.690588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.691889 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.691895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.692858 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.692868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.694115 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.694119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.694200 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.694219 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.694283 (+ 64us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.694348 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":84,"threads_started":1}
I20260621 07:57:03.694592 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.688934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.11.223.129:36911)
0621 07:57:03.689682 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.689695 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.689717 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:57:03.689795 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.689801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.689977 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.690136 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:03.690407 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.691557 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.692896 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.693692 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.693829 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.693943 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.693991 (+ 48us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.694066 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":115,"threads_started":1}
I20260621 07:57:03.694733 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.689410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.11.223.131:38277)
0621 07:57:03.689868 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.689880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.689902 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:57:03.689972 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.689977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.690161 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.690304 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:03.690517 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.691284 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.692858 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.693424 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.693575 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.693703 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.694384 (+ 681us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.694473 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":145,"threads_started":1}
I20260621 07:57:03.694844 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.689275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.11.223.130:33855)
0621 07:57:03.689868 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.689880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.689902 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:57:03.689973 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.689977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.690161 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.690304 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:03.690713 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.691767 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.692975 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.693692 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.693829 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.693942 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.694617 (+ 675us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.694709 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":90,"threads_started":1}
I20260621 07:57:03.697269 19820 tablet_service.cc:1467] Processing CreateTablet for tablet 8c0e30c4e03b48c2a4e114b34a203b55 (DEFAULT_TABLE table=test-table [id=357c0e51e6a34cc39bfe928aba98e642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:03.697288 20100 tablet_service.cc:1467] Processing CreateTablet for tablet 8c0e30c4e03b48c2a4e114b34a203b55 (DEFAULT_TABLE table=test-table [id=357c0e51e6a34cc39bfe928aba98e642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:03.698282 19820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c0e30c4e03b48c2a4e114b34a203b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:03.698293 20100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c0e30c4e03b48c2a4e114b34a203b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:03.698019 19960 tablet_service.cc:1467] Processing CreateTablet for tablet 8c0e30c4e03b48c2a4e114b34a203b55 (DEFAULT_TABLE table=test-table [id=357c0e51e6a34cc39bfe928aba98e642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:03.698839 19960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c0e30c4e03b48c2a4e114b34a203b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:03.704550 20194 tablet_bootstrap.cc:492] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Bootstrap starting.
I20260621 07:57:03.705818 20196 tablet_bootstrap.cc:492] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Bootstrap starting.
I20260621 07:57:03.706087 20195 tablet_bootstrap.cc:492] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Bootstrap starting.
I20260621 07:57:03.706775 20194 tablet_bootstrap.cc:654] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:03.707564 20194 log.cc:824] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:03.707672 20195 tablet_bootstrap.cc:654] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:03.707731 20196 tablet_bootstrap.cc:654] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:03.708452 20195 log.cc:824] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:03.708500 20196 log.cc:824] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:03.709872 20194 tablet_bootstrap.cc:492] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: No bootstrap required, opened a new log
I20260621 07:57:03.710129 20194 ts_tablet_manager.cc:1397] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 07:57:03.710269 20196 tablet_bootstrap.cc:492] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: No bootstrap required, opened a new log
I20260621 07:57:03.710398 20195 tablet_bootstrap.cc:492] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: No bootstrap required, opened a new log
I20260621 07:57:03.710435 20196 ts_tablet_manager.cc:1397] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:57:03.710520 20195 ts_tablet_manager.cc:1397] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260621 07:57:03.713341 20195 raft_consensus.cc:348] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.713578 20195 raft_consensus.cc:374] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:03.713656 20195 raft_consensus.cc:729] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Initialized, Role: FOLLOWER
I20260621 07:57:03.713953 20194 raft_consensus.cc:348] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.713953 20196 raft_consensus.cc:348] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.714110 20195 consensus_queue.cc:260] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.714217 20194 raft_consensus.cc:374] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:03.714228 20196 raft_consensus.cc:374] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:03.714278 20196 raft_consensus.cc:729] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e79c78bee5d494bbdb9925fc808203b, State: Initialized, Role: FOLLOWER
I20260621 07:57:03.714308 20194 raft_consensus.cc:729] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6670959e07544418990c6ab6b835ba8c, State: Initialized, Role: FOLLOWER
I20260621 07:57:03.714713 20196 consensus_queue.cc:260] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.714713 20194 consensus_queue.cc:260] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.715035 20195 ts_tablet_manager.cc:1428] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260621 07:57:03.715339 19886 heartbeater.cc:503] Master 127.11.223.190:46137 was elected leader, sending a full tablet report...
I20260621 07:57:03.715601 20196 ts_tablet_manager.cc:1428] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 07:57:03.715601 20194 ts_tablet_manager.cc:1428] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 07:57:03.715952 20166 heartbeater.cc:503] Master 127.11.223.190:46137 was elected leader, sending a full tablet report...
I20260621 07:57:03.716538 20026 heartbeater.cc:503] Master 127.11.223.190:46137 was elected leader, sending a full tablet report...
W20260621 07:57:03.732661 19887 tablet.cc:2369] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:57:03.886698 20167 tablet.cc:2369] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:57:03.936415 20027 tablet.cc:2369] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:57:03.944489 20201 raft_consensus.cc:482] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:03.944698 20201 raft_consensus.cc:504] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.945742 20201 leader_election.cc:288] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277), 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855)
I20260621 07:57:03.950623 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.946103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52237 (local address 127.11.223.131:38277)
0621 07:57:03.946267 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.946272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.946295 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:57:03.946484 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.946487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.946537 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.946633 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:03.946924 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.947730 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.949315 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.949924 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.950047 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.950096 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.950429 (+ 333us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.950497 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 07:57:03.950620 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.946446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:49437 (local address 127.11.223.130:33855)
0621 07:57:03.946566 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.946572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.946753 (+ 181us) server_negotiation.cc:407] Connection header received
0621 07:57:03.946833 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.946835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.946894 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.946969 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:03.947180 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.947866 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.949315 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.949924 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.950048 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.950107 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.950429 (+ 322us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.950496 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260621 07:57:03.950728 20203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.945980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38277 (local address 127.11.223.129:52237)
0621 07:57:03.946214 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.946227 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.946354 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.946645 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.946649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.946671 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:03.946808 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.946817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.947829 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.947831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.949195 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.949200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.950046 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.950050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.950280 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.950326 (+ 46us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.950401 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.950561 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":172,"thread_start_us":59,"threads_started":1}
I20260621 07:57:03.950731 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.946321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33855 (local address 127.11.223.129:49437)
0621 07:57:03.946660 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:03.946672 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:03.946755 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:03.946983 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:03.946985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:03.946999 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:03.947079 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.947082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.947957 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.947960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.949187 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:03.949198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.950046 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.950050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:03.950284 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.950323 (+ 39us) client_negotiation.cc:769] Sending connection context
0621 07:57:03.950401 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 07:57:03.950564 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":176,"thread_start_us":97,"threads_started":1}
I20260621 07:57:03.951535 19980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0e30c4e03b48c2a4e114b34a203b55" candidate_uuid: "6670959e07544418990c6ab6b835ba8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ed09a4ecea46448a2468c7b95c375b" is_pre_election: true
I20260621 07:57:03.951547 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0e30c4e03b48c2a4e114b34a203b55" candidate_uuid: "6670959e07544418990c6ab6b835ba8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b" is_pre_election: true
I20260621 07:57:03.951805 20120 raft_consensus.cc:2454] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6670959e07544418990c6ab6b835ba8c in term 0.
I20260621 07:57:03.951812 19980 raft_consensus.cc:2454] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6670959e07544418990c6ab6b835ba8c in term 0.
I20260621 07:57:03.952260 19773 leader_election.cc:302] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6670959e07544418990c6ab6b835ba8c; no voters:
I20260621 07:57:03.952548 20201 raft_consensus.cc:2781] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:03.952644 20201 raft_consensus.cc:482] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:03.952696 20201 raft_consensus.cc:3037] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:03.953652 20201 raft_consensus.cc:504] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.953949 20201 leader_election.cc:288] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [CANDIDATE]: Term 1 election: Requested vote from peers 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277), 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855)
I20260621 07:57:03.954289 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0e30c4e03b48c2a4e114b34a203b55" candidate_uuid: "6670959e07544418990c6ab6b835ba8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b"
I20260621 07:57:03.954300 19980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0e30c4e03b48c2a4e114b34a203b55" candidate_uuid: "6670959e07544418990c6ab6b835ba8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ed09a4ecea46448a2468c7b95c375b"
I20260621 07:57:03.954401 19980 raft_consensus.cc:3037] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:03.954402 20120 raft_consensus.cc:3037] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:03.955597 19980 raft_consensus.cc:2454] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6670959e07544418990c6ab6b835ba8c in term 1.
I20260621 07:57:03.955893 19773 leader_election.cc:302] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6670959e07544418990c6ab6b835ba8c; no voters:
I20260621 07:57:03.956051 20201 raft_consensus.cc:2781] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:03.956048 20120 raft_consensus.cc:2454] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6670959e07544418990c6ab6b835ba8c in term 1.
I20260621 07:57:03.956218 20201 raft_consensus.cc:686] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 1 LEADER]: Becoming Leader. State: Replica: 6670959e07544418990c6ab6b835ba8c, State: Running, Role: LEADER
I20260621 07:57:03.956563 20201 consensus_queue.cc:237] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } }
I20260621 07:57:03.959321 19683 catalog_manager.cc:5495] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6670959e07544418990c6ab6b835ba8c (127.11.223.129). New cstate: current_term: 1 leader_uuid: "6670959e07544418990c6ab6b835ba8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } health_report { overall_health: UNKNOWN } } }
I20260621 07:57:03.981626 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:03.977968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.11.223.129:36911)
0621 07:57:03.978080 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:03.978084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:03.978099 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:57:03.978247 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:03.978250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:03.978304 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:03.978404 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:03.978606 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.979284 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.980062 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:03.980439 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:03.980546 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:03.980771 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:03.981229 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:03.981352 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:03.981446 (+ 94us) server_negotiation.cc:299] Negotiation successful
0621 07:57:03.981499 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 07:57:03.987169 20120 raft_consensus.cc:1261] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Refusing update from remote peer 6670959e07544418990c6ab6b835ba8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:03.987185 19980 raft_consensus.cc:1261] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Refusing update from remote peer 6670959e07544418990c6ab6b835ba8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:03.987767 20201 consensus_queue.cc:1035] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:03.987926 20205 consensus_queue.cc:1035] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:03.995029 20210 mvcc.cc:205] Tried to move back new op lower bound from 7299189243845201920 to 7299189243728347136. Current Snapshot: MvccSnapshot[applied={T|T < 7299189243845201920}]
I20260621 07:57:03.995456 20209 mvcc.cc:205] Tried to move back new op lower bound from 7299189243845201920 to 7299189243728347136. Current Snapshot: MvccSnapshot[applied={T|T < 7299189243845201920}]
I20260621 07:57:04.002840 19683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.003005 19683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.005192 19683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Sending DeleteTablet for 3 replicas of tablet 8c0e30c4e03b48c2a4e114b34a203b55
I20260621 07:57:04.005865 19820 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c0e30c4e03b48c2a4e114b34a203b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:03 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:43896
I20260621 07:57:04.006029 20100 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c0e30c4e03b48c2a4e114b34a203b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:03 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:59736
I20260621 07:57:04.006404 19960 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c0e30c4e03b48c2a4e114b34a203b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:03 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260621 07:57:04.006479 20218 tablet_replica.cc:330] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: stopping tablet replica
I20260621 07:57:04.006453 19683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260621 07:57:04.006745 20218 raft_consensus.cc:2229] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 1 LEADER]: Raft consensus shutting down.
W20260621 07:57:04.006820 19683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:04.007078 20218 raft_consensus.cc:2258] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.007411 20219 tablet_replica.cc:330] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: stopping tablet replica
I20260621 07:57:04.008397 20219 raft_consensus.cc:2229] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.008473 20218 ts_tablet_manager.cc:1905] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.008812 20219 raft_consensus.cc:2258] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.010880 20219 ts_tablet_manager.cc:1905] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.011436 19820 tablet_service.cc:1467] Processing CreateTablet for tablet 854b31d079ab4fb5b4f02d99db78434d (DEFAULT_TABLE table=test-table [id=fd7faade2a654fd894e2f8bac202b443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.011474 20218 ts_tablet_manager.cc:1918] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:04.011590 20218 log.cc:1196] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/8c0e30c4e03b48c2a4e114b34a203b55
I20260621 07:57:04.011713 19820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854b31d079ab4fb5b4f02d99db78434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.011711 19960 tablet_service.cc:1467] Processing CreateTablet for tablet 854b31d079ab4fb5b4f02d99db78434d (DEFAULT_TABLE table=test-table [id=fd7faade2a654fd894e2f8bac202b443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.011904 20218 ts_tablet_manager.cc:1939] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6670959e07544418990c6ab6b835ba8c: Deleting consensus metadata
I20260621 07:57:04.011991 19960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854b31d079ab4fb5b4f02d99db78434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.012960 19670 catalog_manager.cc:4841] TS 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911): tablet 8c0e30c4e03b48c2a4e114b34a203b55 (table test-table [id=357c0e51e6a34cc39bfe928aba98e642]) successfully deleted
I20260621 07:57:04.013777 20194 tablet_bootstrap.cc:492] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Bootstrap starting.
I20260621 07:57:04.014180 20219 ts_tablet_manager.cc:1918] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:04.014243 20195 tablet_bootstrap.cc:492] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Bootstrap starting.
I20260621 07:57:04.014264 20219 log.cc:1196] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/8c0e30c4e03b48c2a4e114b34a203b55
I20260621 07:57:04.014587 20219 ts_tablet_manager.cc:1939] T 8c0e30c4e03b48c2a4e114b34a203b55 P 6e79c78bee5d494bbdb9925fc808203b: Deleting consensus metadata
I20260621 07:57:04.014647 20194 tablet_bootstrap.cc:654] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.015228 20195 tablet_bootstrap.cc:654] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.015475 20100 tablet_service.cc:1467] Processing CreateTablet for tablet 854b31d079ab4fb5b4f02d99db78434d (DEFAULT_TABLE table=test-table [id=fd7faade2a654fd894e2f8bac202b443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.015642 19669 catalog_manager.cc:4841] TS 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277): tablet 8c0e30c4e03b48c2a4e114b34a203b55 (table test-table [id=357c0e51e6a34cc39bfe928aba98e642]) successfully deleted
I20260621 07:57:04.015725 20100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854b31d079ab4fb5b4f02d99db78434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.015946 20220 tablet_replica.cc:330] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: stopping tablet replica
I20260621 07:57:04.016232 20220 raft_consensus.cc:2229] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.016500 20220 raft_consensus.cc:2258] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.017565 20194 tablet_bootstrap.cc:492] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: No bootstrap required, opened a new log
I20260621 07:57:04.017647 20194 ts_tablet_manager.cc:1397] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260621 07:57:04.017751 20195 tablet_bootstrap.cc:492] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: No bootstrap required, opened a new log
I20260621 07:57:04.017827 20195 ts_tablet_manager.cc:1397] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260621 07:57:04.017868 20196 tablet_bootstrap.cc:492] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Bootstrap starting.
I20260621 07:57:04.018080 20194 raft_consensus.cc:348] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.018187 20194 raft_consensus.cc:374] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.018224 20194 raft_consensus.cc:729] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6670959e07544418990c6ab6b835ba8c, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.018234 20195 raft_consensus.cc:348] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.018325 20195 raft_consensus.cc:374] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.018353 20195 raft_consensus.cc:729] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.018339 20194 consensus_queue.cc:260] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.018458 20195 consensus_queue.cc:260] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.018571 20194 ts_tablet_manager.cc:1428] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:04.018697 20195 ts_tablet_manager.cc:1428] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:57:04.018764 20196 tablet_bootstrap.cc:654] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.019794 20196 tablet_bootstrap.cc:492] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: No bootstrap required, opened a new log
I20260621 07:57:04.019870 20196 ts_tablet_manager.cc:1397] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:04.020285 20196 raft_consensus.cc:348] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.020459 20196 raft_consensus.cc:374] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.020500 20196 raft_consensus.cc:729] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e79c78bee5d494bbdb9925fc808203b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.020622 20196 consensus_queue.cc:260] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.020857 20196 ts_tablet_manager.cc:1428] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:04.021981 20220 ts_tablet_manager.cc:1905] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.025108 20220 ts_tablet_manager.cc:1918] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:04.025178 20220 log.cc:1196] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/8c0e30c4e03b48c2a4e114b34a203b55
I20260621 07:57:04.025472 20220 ts_tablet_manager.cc:1939] T 8c0e30c4e03b48c2a4e114b34a203b55 P 12ed09a4ecea46448a2468c7b95c375b: Deleting consensus metadata
I20260621 07:57:04.026381 19669 catalog_manager.cc:4841] TS 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855): tablet 8c0e30c4e03b48c2a4e114b34a203b55 (table test-table [id=357c0e51e6a34cc39bfe928aba98e642]) successfully deleted
I20260621 07:57:04.053484 20202 raft_consensus.cc:482] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:04.053637 20202 raft_consensus.cc:504] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.054636 20202 leader_election.cc:288] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855)
I20260621 07:57:04.058894 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.055089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:41737 (local address 127.11.223.129:36911)
0621 07:57:04.055199 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.055206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.055222 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:57:04.055268 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.055271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.055330 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.055410 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:04.055792 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.056581 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.058027 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.058384 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.058505 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.058554 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.058729 (+ 175us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.058788 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 07:57:04.058913 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.054784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36911 (local address 127.11.223.131:41737)
0621 07:57:04.055037 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.055051 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.055133 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.055523 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.055529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.055548 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:04.055669 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.055678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.056692 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.056695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.057903 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.057913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.058498 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.058502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.058603 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.058630 (+ 27us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.058715 (+ 85us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.058780 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260621 07:57:04.059491 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854b31d079ab4fb5b4f02d99db78434d" candidate_uuid: "6e79c78bee5d494bbdb9925fc808203b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c" is_pre_election: true
I20260621 07:57:04.059708 19840 raft_consensus.cc:2454] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e79c78bee5d494bbdb9925fc808203b in term 0.
I20260621 07:57:04.060144 20054 leader_election.cc:302] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6670959e07544418990c6ab6b835ba8c, 6e79c78bee5d494bbdb9925fc808203b; no voters:
I20260621 07:57:04.060403 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.054852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33855 (local address 127.11.223.131:45111)
0621 07:57:04.055150 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.055164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.055274 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.055602 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.055605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.055617 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:04.055713 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.055722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.058035 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.058038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.059178 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.059188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.059945 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.059948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.060051 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.060075 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.060177 (+ 102us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.060240 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":71,"threads_started":1}
I20260621 07:57:04.060416 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.054986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:45111 (local address 127.11.223.130:33855)
0621 07:57:04.055278 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.055283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.055306 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:57:04.055366 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.055369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.055406 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.055460 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:04.055832 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.056586 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.059328 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.059789 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.059929 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.059982 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.060208 (+ 226us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.060284 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260621 07:57:04.060415 20202 raft_consensus.cc:2781] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:04.060817 20202 raft_consensus.cc:482] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:04.060804 19980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854b31d079ab4fb5b4f02d99db78434d" candidate_uuid: "6e79c78bee5d494bbdb9925fc808203b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ed09a4ecea46448a2468c7b95c375b" is_pre_election: true
I20260621 07:57:04.060868 20202 raft_consensus.cc:3037] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.060926 19980 raft_consensus.cc:2454] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e79c78bee5d494bbdb9925fc808203b in term 0.
I20260621 07:57:04.061874 20202 raft_consensus.cc:504] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.062225 20202 leader_election.cc:288] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [CANDIDATE]: Term 1 election: Requested vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855)
I20260621 07:57:04.062511 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854b31d079ab4fb5b4f02d99db78434d" candidate_uuid: "6e79c78bee5d494bbdb9925fc808203b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c"
I20260621 07:57:04.062615 19840 raft_consensus.cc:3037] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.062645 19980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854b31d079ab4fb5b4f02d99db78434d" candidate_uuid: "6e79c78bee5d494bbdb9925fc808203b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ed09a4ecea46448a2468c7b95c375b"
I20260621 07:57:04.062733 19980 raft_consensus.cc:3037] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.063562 19980 raft_consensus.cc:2454] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e79c78bee5d494bbdb9925fc808203b in term 1.
I20260621 07:57:04.063683 19840 raft_consensus.cc:2454] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e79c78bee5d494bbdb9925fc808203b in term 1.
I20260621 07:57:04.063858 20053 leader_election.cc:302] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6e79c78bee5d494bbdb9925fc808203b; no voters:
I20260621 07:57:04.064005 20202 raft_consensus.cc:2781] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:04.064236 20202 raft_consensus.cc:686] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 1 LEADER]: Becoming Leader. State: Replica: 6e79c78bee5d494bbdb9925fc808203b, State: Running, Role: LEADER
I20260621 07:57:04.064507 20202 consensus_queue.cc:237] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.066442 19683 catalog_manager.cc:5495] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131). New cstate: current_term: 1 leader_uuid: "6e79c78bee5d494bbdb9925fc808203b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } health_report { overall_health: HEALTHY } } }
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028622, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.190@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:04.110335 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.101768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46137 (local address 127.0.0.1:38762)
0621 07:57:04.102023 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.102034 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.102145 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.102391 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.102393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.102750 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:04.102825 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.102831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.103573 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.103576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.104203 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.104216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.104866 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.104868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.104929 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.105474 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:04.105484 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:04.106926 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:04.108738 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:04.108747 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:04.108752 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:04.108973 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:04.109259 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:04.109261 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:04.109263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:04.109337 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:04.109717 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:04.109725 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:04.109825 (+ 100us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.109933 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.110079 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":85,"threads_started":1}
I20260621 07:57:04.110381 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.101828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.11.223.190:46137)
0621 07:57:04.101945 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.101950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.102107 (+ 157us) server_negotiation.cc:407] Connection header received
0621 07:57:04.102242 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.102246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.102288 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.102377 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:04.102987 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.103465 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.104347 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.104755 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.104861 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.107067 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:04.107070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:04.107073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:04.107098 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:04.108630 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:04.109079 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:04.109082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:04.109084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:04.109171 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:04.109423 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:04.109425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:04.109427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:04.109607 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:04.109708 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.109985 (+ 277us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.110139 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":41}
I20260621 07:57:04.114024 19683 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:38762:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260621 07:57:04.114751 19683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:04.119351 19820 tablet_service.cc:1467] Processing CreateTablet for tablet 64768ef71cea4415999258b0ff651310 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80271c8819dc4c5f9729c3e7b70c35de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:57:04.119351 19960 tablet_service.cc:1467] Processing CreateTablet for tablet 64768ef71cea4415999258b0ff651310 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80271c8819dc4c5f9729c3e7b70c35de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:57:04.119678 19960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64768ef71cea4415999258b0ff651310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.119678 19820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64768ef71cea4415999258b0ff651310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.122373 20195 tablet_bootstrap.cc:492] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: Bootstrap starting.
I20260621 07:57:04.122691 20100 tablet_service.cc:1467] Processing CreateTablet for tablet 64768ef71cea4415999258b0ff651310 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80271c8819dc4c5f9729c3e7b70c35de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260621 07:57:04.123018 20100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64768ef71cea4415999258b0ff651310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.123338 20195 tablet_bootstrap.cc:654] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.124579 20195 tablet_bootstrap.cc:492] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: No bootstrap required, opened a new log
I20260621 07:57:04.124666 20195 ts_tablet_manager.cc:1397] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260621 07:57:04.125074 20195 raft_consensus.cc:348] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.125190 20195 raft_consensus.cc:374] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.125221 20195 raft_consensus.cc:729] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.125332 20195 consensus_queue.cc:260] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.125589 20195 ts_tablet_manager.cc:1428] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:57:04.125766 20196 tablet_bootstrap.cc:492] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: Bootstrap starting.
I20260621 07:57:04.125929 20194 tablet_bootstrap.cc:492] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: Bootstrap starting.
I20260621 07:57:04.126621 20194 tablet_bootstrap.cc:654] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.126652 20196 tablet_bootstrap.cc:654] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.127702 20194 tablet_bootstrap.cc:492] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: No bootstrap required, opened a new log
I20260621 07:57:04.127650 20200 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.127781 20194 ts_tablet_manager.cc:1397] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:04.127795 20200 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.128153 20194 raft_consensus.cc:348] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.128274 20194 raft_consensus.cc:374] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.128360 20194 raft_consensus.cc:729] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6670959e07544418990c6ab6b835ba8c, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.128497 20194 consensus_queue.cc:260] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.128645 20196 tablet_bootstrap.cc:492] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: No bootstrap required, opened a new log
I20260621 07:57:04.128723 20196 ts_tablet_manager.cc:1397] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260621 07:57:04.128769 20194 ts_tablet_manager.cc:1428] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:04.128961 20214 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.129112 20214 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.129375 20196 raft_consensus.cc:348] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.129527 20196 raft_consensus.cc:374] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.129586 20196 raft_consensus.cc:729] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e79c78bee5d494bbdb9925fc808203b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.129727 20196 consensus_queue.cc:260] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.130004 20196 ts_tablet_manager.cc:1428] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260621 07:57:04.130510 20202 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.130642 20202 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.277000 20200 raft_consensus.cc:482] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:04.277133 20200 raft_consensus.cc:504] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.278100 20200 leader_election.cc:288] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:04.282040 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.278405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:41755 (local address 127.11.223.129:36911)
0621 07:57:04.278526 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.278530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.278689 (+ 159us) server_negotiation.cc:407] Connection header received
0621 07:57:04.278837 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.278840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.278886 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.278975 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:04.279272 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.279759 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.281086 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.281463 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.281618 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.281691 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.281861 (+ 170us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.281923 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:57:04.282045 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.278394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:55755 (local address 127.11.223.131:38277)
0621 07:57:04.278524 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.278530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.278724 (+ 194us) server_negotiation.cc:407] Connection header received
0621 07:57:04.278877 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.278880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.278918 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.278999 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:04.279298 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.279762 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.281129 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.281504 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.281667 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.281738 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.281874 (+ 136us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.281938 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:57:04.282071 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.278283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36911 (local address 127.11.223.130:41755)
0621 07:57:04.278604 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.278616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.278738 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.278986 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.278989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.279003 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:04.279169 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.279175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.279869 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.279873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.280968 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.280981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.281591 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.281603 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.281722 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.281749 (+ 27us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.281838 (+ 89us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.281910 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":117,"spinlock_wait_cycles":2432,"thread_start_us":91,"threads_started":1}
I20260621 07:57:04.282079 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.278304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38277 (local address 127.11.223.130:55755)
0621 07:57:04.278637 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.278648 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.278762 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.279008 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.279011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.279046 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:04.279194 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.279199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.279864 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.279867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.281002 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.281012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.281659 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.281662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.281760 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.281787 (+ 27us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.281855 (+ 68us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.281920 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":70,"spinlock_wait_cycles":20608,"thread_start_us":100,"threads_started":1}
I20260621 07:57:04.282711 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64768ef71cea4415999258b0ff651310" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b" is_pre_election: true
I20260621 07:57:04.282851 20120 raft_consensus.cc:2454] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 0.
I20260621 07:57:04.283072 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64768ef71cea4415999258b0ff651310" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c" is_pre_election: true
I20260621 07:57:04.283205 19840 raft_consensus.cc:2454] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 0.
I20260621 07:57:04.283174 19913 leader_election.cc:302] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6e79c78bee5d494bbdb9925fc808203b; no voters:
I20260621 07:57:04.283448 20200 raft_consensus.cc:2781] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:04.283496 20200 raft_consensus.cc:482] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:04.283542 20200 raft_consensus.cc:3037] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.284461 20200 raft_consensus.cc:504] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.284801 20200 leader_election.cc:288] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 election: Requested vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:04.285089 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64768ef71cea4415999258b0ff651310" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b"
I20260621 07:57:04.285099 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64768ef71cea4415999258b0ff651310" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c"
I20260621 07:57:04.285207 20120 raft_consensus.cc:3037] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.285207 19840 raft_consensus.cc:3037] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.286307 20120 raft_consensus.cc:2454] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 1.
I20260621 07:57:04.286307 19840 raft_consensus.cc:2454] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 1.
I20260621 07:57:04.286599 19914 leader_election.cc:302] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6670959e07544418990c6ab6b835ba8c; no voters:
I20260621 07:57:04.286753 20200 raft_consensus.cc:2781] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:04.286973 20200 raft_consensus.cc:686] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [term 1 LEADER]: Becoming Leader. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Running, Role: LEADER
I20260621 07:57:04.287222 20200 consensus_queue.cc:237] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.288370 20238 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: TxnStatusTablet state changed. Reason: New leader 12ed09a4ecea46448a2468c7b95c375b. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.288487 20238 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:57:04.288733 20240 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260621 07:57:04.288882 20240 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260621 07:57:04.289544 19682 catalog_manager.cc:5495] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b reported cstate change: term changed from 0 to 1, leader changed from <none> to 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130). New cstate: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } health_report { overall_health: UNKNOWN } } }
I20260621 07:57:04.322036 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.318560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.11.223.130:33855)
0621 07:57:04.318683 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.318688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.318703 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:57:04.318843 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.318846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.318897 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.318998 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:04.319249 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.319727 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.320648 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.320869 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.320976 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.321272 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.321663 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.321783 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.321848 (+ 65us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.321896 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:57:04.322036 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.318426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33855 (local address 127.0.0.1:57756)
0621 07:57:04.318565 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.318582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.318684 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.319011 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.319014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.319021 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:57:04.319138 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.319143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.319835 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.319839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.320526 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.320535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.320977 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.320980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.321071 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.321117 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.321758 (+ 641us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.321773 (+ 15us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.321840 (+ 67us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.321920 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260621 07:57:04.324827 20120 raft_consensus.cc:1261] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Refusing update from remote peer 12ed09a4ecea46448a2468c7b95c375b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.325006 19840 raft_consensus.cc:1261] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Refusing update from remote peer 12ed09a4ecea46448a2468c7b95c375b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.325291 20238 consensus_queue.cc:1035] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.325548 20200 consensus_queue.cc:1035] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.326614 20202 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: TxnStatusTablet state changed. Reason: New leader 12ed09a4ecea46448a2468c7b95c375b. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.326726 20202 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.327167 20214 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: TxnStatusTablet state changed. Reason: New leader 12ed09a4ecea46448a2468c7b95c375b. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.327273 20214 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.329839 20200 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.329949 20200 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:57:04.329882 20245 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.330195 20245 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 12ed09a4ecea46448a2468c7b95c375b: This TxnStatusTablet replica's current role is: LEADER
I20260621 07:57:04.330670 20208 mvcc.cc:205] Tried to move back new op lower bound from 7299189245229535232 to 7299189245082861568. Current Snapshot: MvccSnapshot[applied={T|T < 7299189245229535232}]
I20260621 07:57:04.330837 20214 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.330920 20214 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 6670959e07544418990c6ab6b835ba8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.332773 20202 tablet_replica.cc:439] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } } }
I20260621 07:57:04.332887 20202 tablet_replica.cc:442] T 64768ef71cea4415999258b0ff651310 P 6e79c78bee5d494bbdb9925fc808203b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260621 07:57:04.337308 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.333388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.11.223.131:38277)
0621 07:57:04.333488 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.333493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.333582 (+ 89us) server_negotiation.cc:407] Connection header received
0621 07:57:04.333743 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.333745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.333801 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.333899 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:04.334196 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.334930 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.335756 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.336078 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.336214 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.336478 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.336957 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.337057 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.337128 (+ 71us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.337177 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260621 07:57:04.347105 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.343779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:44941 (local address 127.11.223.130:33855)
0621 07:57:04.343891 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.343895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.344137 (+ 242us) server_negotiation.cc:407] Connection header received
0621 07:57:04.344246 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.344248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.344284 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.344406 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:04.344663 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.345116 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.346227 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.346416 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.346483 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.346765 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.346819 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.346863 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.346978 (+ 115us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.347014 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 07:57:04.347175 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.343680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33855 (local address 127.11.223.131:44941)
0621 07:57:04.344049 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.344063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.344166 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.344439 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.344442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.344450 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:57:04.344556 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.344562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.345219 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.345222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.346129 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.346138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.346510 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.346514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.346605 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.346648 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.346900 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.346905 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.346965 (+ 60us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.347030 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":130,"threads_started":1}
I20260621 07:57:04.353684 19840 raft_consensus.cc:1261] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Refusing update from remote peer 6e79c78bee5d494bbdb9925fc808203b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.353682 19980 raft_consensus.cc:1261] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Refusing update from remote peer 6e79c78bee5d494bbdb9925fc808203b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.354130 20225 consensus_queue.cc:1035] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.354501 20225 consensus_queue.cc:1035] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.376055 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.372275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:38277 (local address 127.11.223.130:42209)
0621 07:57:04.372589 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.372603 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.372700 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.372911 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.372913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.372917 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:57:04.373002 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.373005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.373937 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.373940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.374805 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.374815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.375429 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.375433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.375507 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.375544 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.375829 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.375834 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.375880 (+ 46us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.375935 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":109,"threads_started":1}
I20260621 07:57:04.376051 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.372588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:42209 (local address 127.11.223.131:38277)
0621 07:57:04.372685 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.372690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.372703 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:57:04.372768 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.372771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.372814 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.372905 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:04.373080 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.373811 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.374912 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.375236 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.375339 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.375672 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.375747 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.375821 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.375889 (+ 68us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.375935 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260621 07:57:04.398362 19683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.398518 19683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.400691 19683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Sending DeleteTablet for 3 replicas of tablet 854b31d079ab4fb5b4f02d99db78434d
I20260621 07:57:04.401186 19820 tablet_service.cc:1514] Processing DeleteTablet for tablet 854b31d079ab4fb5b4f02d99db78434d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:43896
I20260621 07:57:04.401371 20218 tablet_replica.cc:330] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: stopping tablet replica
I20260621 07:57:04.401475 20218 raft_consensus.cc:2229] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.401573 20218 raft_consensus.cc:2258] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.402001 20100 tablet_service.cc:1514] Processing DeleteTablet for tablet 854b31d079ab4fb5b4f02d99db78434d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:59736
I20260621 07:57:04.402163 20219 tablet_replica.cc:330] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: stopping tablet replica
I20260621 07:57:04.402283 20219 raft_consensus.cc:2229] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:57:04.402529 20218 ts_tablet_manager.cc:1905] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.402565 20219 raft_consensus.cc:2258] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.403371 20219 ts_tablet_manager.cc:1905] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.405398 20218 ts_tablet_manager.cc:1918] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:57:04.405462 20218 log.cc:1196] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/854b31d079ab4fb5b4f02d99db78434d
I20260621 07:57:04.405737 20218 ts_tablet_manager.cc:1939] T 854b31d079ab4fb5b4f02d99db78434d P 6670959e07544418990c6ab6b835ba8c: Deleting consensus metadata
I20260621 07:57:04.406167 20219 ts_tablet_manager.cc:1918] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:57:04.406231 20219 log.cc:1196] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/854b31d079ab4fb5b4f02d99db78434d
I20260621 07:57:04.406405 19670 catalog_manager.cc:4841] TS 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911): tablet 854b31d079ab4fb5b4f02d99db78434d (table test-table [id=fd7faade2a654fd894e2f8bac202b443]) successfully deleted
I20260621 07:57:04.406476 20219 ts_tablet_manager.cc:1939] T 854b31d079ab4fb5b4f02d99db78434d P 6e79c78bee5d494bbdb9925fc808203b: Deleting consensus metadata
I20260621 07:57:04.407104 19669 catalog_manager.cc:4841] TS 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277): tablet 854b31d079ab4fb5b4f02d99db78434d (table test-table [id=fd7faade2a654fd894e2f8bac202b443]) successfully deleted
I20260621 07:57:04.407513 19960 tablet_service.cc:1514] Processing DeleteTablet for tablet 854b31d079ab4fb5b4f02d99db78434d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260621 07:57:04.407701 20220 tablet_replica.cc:330] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: stopping tablet replica
I20260621 07:57:04.407829 20220 raft_consensus.cc:2229] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.407951 20220 raft_consensus.cc:2258] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.408735 20220 ts_tablet_manager.cc:1905] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.411231 20220 ts_tablet_manager.cc:1918] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:57:04.411304 20220 log.cc:1196] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/854b31d079ab4fb5b4f02d99db78434d
I20260621 07:57:04.411526 20220 ts_tablet_manager.cc:1939] T 854b31d079ab4fb5b4f02d99db78434d P 12ed09a4ecea46448a2468c7b95c375b: Deleting consensus metadata
I20260621 07:57:04.412190 19669 catalog_manager.cc:4841] TS 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855): tablet 854b31d079ab4fb5b4f02d99db78434d (table test-table [id=fd7faade2a654fd894e2f8bac202b443]) successfully deleted
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
W20260621 07:57:04.439635 19732 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:04.431961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.11.223.190:46137)
0621 07:57:04.432081 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.432085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.432382 (+ 297us) server_negotiation.cc:407] Connection header received
0621 07:57:04.432591 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.432594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.432652 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.432746 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:04.433557 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.434382 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.435586 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.435900 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.435983 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.439502 (+ 3519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38778: BlockingRecv error: recv got EOF from 127.0.0.1:38778 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM
I20260621 07:57:04.451473 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.441534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.11.223.190:46137)
0621 07:57:04.441639 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.441643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.442109 (+ 466us) server_negotiation.cc:407] Connection header received
0621 07:57:04.442267 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.442270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.442310 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.442355 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:04.443127 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.443940 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.445085 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.445414 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.445475 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.448161 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:04.448164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:04.448167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:04.448192 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:04.449759 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:04.450216 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:04.450220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:04.450221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:04.450300 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:04.450595 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:04.450599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:04.450600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:04.450791 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:04.450888 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.451217 (+ 329us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.451333 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260621 07:57:04.454504 19683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:57:04.454958 19683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:04.460564 20100 tablet_service.cc:1467] Processing CreateTablet for tablet 12309d5fef3f43239b5a9ed0ce4de75d (DEFAULT_TABLE table=test-table [id=30c3fc667cc346518b2ad3f475bc9bb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.460865 20100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12309d5fef3f43239b5a9ed0ce4de75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.461651 19820 tablet_service.cc:1467] Processing CreateTablet for tablet 12309d5fef3f43239b5a9ed0ce4de75d (DEFAULT_TABLE table=test-table [id=30c3fc667cc346518b2ad3f475bc9bb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.461891 19820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12309d5fef3f43239b5a9ed0ce4de75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.463219 20196 tablet_bootstrap.cc:492] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Bootstrap starting.
I20260621 07:57:04.464018 20194 tablet_bootstrap.cc:492] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Bootstrap starting.
I20260621 07:57:04.464138 20196 tablet_bootstrap.cc:654] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.464818 20194 tablet_bootstrap.cc:654] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.464874 19960 tablet_service.cc:1467] Processing CreateTablet for tablet 12309d5fef3f43239b5a9ed0ce4de75d (DEFAULT_TABLE table=test-table [id=30c3fc667cc346518b2ad3f475bc9bb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.465101 19960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12309d5fef3f43239b5a9ed0ce4de75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.466842 20194 tablet_bootstrap.cc:492] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: No bootstrap required, opened a new log
I20260621 07:57:04.466917 20194 ts_tablet_manager.cc:1397] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260621 07:57:04.467206 20195 tablet_bootstrap.cc:492] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Bootstrap starting.
I20260621 07:57:04.467284 20194 raft_consensus.cc:348] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.467381 20194 raft_consensus.cc:374] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.467407 20194 raft_consensus.cc:729] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6670959e07544418990c6ab6b835ba8c, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.467521 20194 consensus_queue.cc:260] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.467764 20194 ts_tablet_manager.cc:1428] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:04.468040 20195 tablet_bootstrap.cc:654] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.469280 20196 tablet_bootstrap.cc:492] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: No bootstrap required, opened a new log
I20260621 07:57:04.469359 20196 ts_tablet_manager.cc:1397] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260621 07:57:04.469805 20196 raft_consensus.cc:348] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.469902 20196 raft_consensus.cc:374] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.469931 20196 raft_consensus.cc:729] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e79c78bee5d494bbdb9925fc808203b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.470041 20196 consensus_queue.cc:260] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.470278 20196 ts_tablet_manager.cc:1428] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:57:04.472158 20195 tablet_bootstrap.cc:492] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: No bootstrap required, opened a new log
I20260621 07:57:04.472271 20195 ts_tablet_manager.cc:1397] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260621 07:57:04.472693 20195 raft_consensus.cc:348] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.472812 20195 raft_consensus.cc:374] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.472860 20195 raft_consensus.cc:729] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.472980 20195 consensus_queue.cc:260] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.473259 20195 ts_tablet_manager.cc:1428] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:04.645062 20246 raft_consensus.cc:482] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:04.645200 20246 raft_consensus.cc:504] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.645620 20246 leader_election.cc:288] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:04.645989 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12309d5fef3f43239b5a9ed0ce4de75d" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b" is_pre_election: true
I20260621 07:57:04.646011 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12309d5fef3f43239b5a9ed0ce4de75d" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c" is_pre_election: true
I20260621 07:57:04.646137 20120 raft_consensus.cc:2454] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 0.
I20260621 07:57:04.646159 19840 raft_consensus.cc:2454] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 0.
I20260621 07:57:04.646418 19913 leader_election.cc:302] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6e79c78bee5d494bbdb9925fc808203b; no voters:
I20260621 07:57:04.646588 20246 raft_consensus.cc:2781] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:04.646669 20246 raft_consensus.cc:482] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:04.646720 20246 raft_consensus.cc:3037] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.647697 20246 raft_consensus.cc:504] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.648028 20246 leader_election.cc:288] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 election: Requested vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:04.648281 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12309d5fef3f43239b5a9ed0ce4de75d" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c"
I20260621 07:57:04.648339 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12309d5fef3f43239b5a9ed0ce4de75d" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b"
I20260621 07:57:04.648432 19840 raft_consensus.cc:3037] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.648437 20120 raft_consensus.cc:3037] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.649231 20120 raft_consensus.cc:2454] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 1.
I20260621 07:57:04.649449 19840 raft_consensus.cc:2454] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 1.
I20260621 07:57:04.649528 19913 leader_election.cc:302] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6e79c78bee5d494bbdb9925fc808203b; no voters:
I20260621 07:57:04.649706 20246 raft_consensus.cc:2781] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:04.649803 20246 raft_consensus.cc:686] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 1 LEADER]: Becoming Leader. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Running, Role: LEADER
I20260621 07:57:04.649919 20246 consensus_queue.cc:237] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.651443 19682 catalog_manager.cc:5495] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b reported cstate change: term changed from 0 to 1, leader changed from <none> to 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130). New cstate: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } health_report { overall_health: UNKNOWN } } }
I20260621 07:57:04.662302 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.659159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.11.223.130:33855)
0621 07:57:04.659264 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.659268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.659326 (+ 58us) server_negotiation.cc:407] Connection header received
0621 07:57:04.659491 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.659495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.659543 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.659629 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:04.659865 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.660437 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.661277 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.661495 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.661610 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.661865 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.661928 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.662009 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.662125 (+ 116us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.662173 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 07:57:04.664410 19840 raft_consensus.cc:1261] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Refusing update from remote peer 12ed09a4ecea46448a2468c7b95c375b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.664462 20120 raft_consensus.cc:1261] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Refusing update from remote peer 12ed09a4ecea46448a2468c7b95c375b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.664786 20246 consensus_queue.cc:1035] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.664927 20238 consensus_queue.cc:1035] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.674690 19683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.674820 19683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.676607 19683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Sending DeleteTablet for 3 replicas of tablet 12309d5fef3f43239b5a9ed0ce4de75d
I20260621 07:57:04.677091 19820 tablet_service.cc:1514] Processing DeleteTablet for tablet 12309d5fef3f43239b5a9ed0ce4de75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:43896
I20260621 07:57:04.677235 20100 tablet_service.cc:1514] Processing DeleteTablet for tablet 12309d5fef3f43239b5a9ed0ce4de75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:59736
I20260621 07:57:04.677474 20218 tablet_replica.cc:330] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: stopping tablet replica
I20260621 07:57:04.677629 20218 raft_consensus.cc:2229] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.677781 20218 raft_consensus.cc:2258] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.677819 19683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260621 07:57:04.677951 20219 tablet_replica.cc:330] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: stopping tablet replica
I20260621 07:57:04.678056 20219 raft_consensus.cc:2229] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.678181 20219 raft_consensus.cc:2258] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260621 07:57:04.678196 19683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:04.678675 19960 tablet_service.cc:1514] Processing DeleteTablet for tablet 12309d5fef3f43239b5a9ed0ce4de75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260621 07:57:04.678826 20220 tablet_replica.cc:330] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: stopping tablet replica
I20260621 07:57:04.678915 20220 raft_consensus.cc:2229] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:57:04.679157 20220 raft_consensus.cc:2258] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.679217 20218 ts_tablet_manager.cc:1905] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.679894 20220 ts_tablet_manager.cc:1905] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.681639 20218 ts_tablet_manager.cc:1918] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:04.681715 20218 log.cc:1196] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/12309d5fef3f43239b5a9ed0ce4de75d
I20260621 07:57:04.681955 20218 ts_tablet_manager.cc:1939] T 12309d5fef3f43239b5a9ed0ce4de75d P 6670959e07544418990c6ab6b835ba8c: Deleting consensus metadata
I20260621 07:57:04.682337 20220 ts_tablet_manager.cc:1918] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:04.682410 20220 log.cc:1196] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/12309d5fef3f43239b5a9ed0ce4de75d
I20260621 07:57:04.682577 19670 catalog_manager.cc:4841] TS 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911): tablet 12309d5fef3f43239b5a9ed0ce4de75d (table test-table [id=30c3fc667cc346518b2ad3f475bc9bb1]) successfully deleted
I20260621 07:57:04.682678 20220 ts_tablet_manager.cc:1939] T 12309d5fef3f43239b5a9ed0ce4de75d P 12ed09a4ecea46448a2468c7b95c375b: Deleting consensus metadata
I20260621 07:57:04.682981 20219 ts_tablet_manager.cc:1905] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.683287 19669 catalog_manager.cc:4841] TS 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855): tablet 12309d5fef3f43239b5a9ed0ce4de75d (table test-table [id=30c3fc667cc346518b2ad3f475bc9bb1]) successfully deleted
I20260621 07:57:04.685066 20219 ts_tablet_manager.cc:1918] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:04.685201 20219 log.cc:1196] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/12309d5fef3f43239b5a9ed0ce4de75d
I20260621 07:57:04.685070 20100 tablet_service.cc:1467] Processing CreateTablet for tablet ec19a6dacb834d99b5170ac0b781c220 (DEFAULT_TABLE table=test-table [id=c92df2f190554ee9bc72bfd35f7b6661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.685384 20100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec19a6dacb834d99b5170ac0b781c220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.685374 19960 tablet_service.cc:1467] Processing CreateTablet for tablet ec19a6dacb834d99b5170ac0b781c220 (DEFAULT_TABLE table=test-table [id=c92df2f190554ee9bc72bfd35f7b6661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.685431 20219 ts_tablet_manager.cc:1939] T 12309d5fef3f43239b5a9ed0ce4de75d P 6e79c78bee5d494bbdb9925fc808203b: Deleting consensus metadata
I20260621 07:57:04.685580 19960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec19a6dacb834d99b5170ac0b781c220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.687141 20196 tablet_bootstrap.cc:492] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Bootstrap starting.
I20260621 07:57:04.687470 20195 tablet_bootstrap.cc:492] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Bootstrap starting.
I20260621 07:57:04.687637 19669 catalog_manager.cc:4841] TS 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277): tablet 12309d5fef3f43239b5a9ed0ce4de75d (table test-table [id=30c3fc667cc346518b2ad3f475bc9bb1]) successfully deleted
I20260621 07:57:04.687904 20196 tablet_bootstrap.cc:654] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.688210 20195 tablet_bootstrap.cc:654] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.688890 19820 tablet_service.cc:1467] Processing CreateTablet for tablet ec19a6dacb834d99b5170ac0b781c220 (DEFAULT_TABLE table=test-table [id=c92df2f190554ee9bc72bfd35f7b6661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:04.688956 20196 tablet_bootstrap.cc:492] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: No bootstrap required, opened a new log
I20260621 07:57:04.689025 20196 ts_tablet_manager.cc:1397] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260621 07:57:04.689105 19820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec19a6dacb834d99b5170ac0b781c220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:04.689155 20195 tablet_bootstrap.cc:492] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: No bootstrap required, opened a new log
I20260621 07:57:04.689217 20195 ts_tablet_manager.cc:1397] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:04.689400 20196 raft_consensus.cc:348] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.689530 20196 raft_consensus.cc:374] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.689568 20196 raft_consensus.cc:729] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e79c78bee5d494bbdb9925fc808203b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.689569 20195 raft_consensus.cc:348] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.689692 20195 raft_consensus.cc:374] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.689724 20195 raft_consensus.cc:729] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.689685 20196 consensus_queue.cc:260] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.689829 20195 consensus_queue.cc:260] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.689946 20196 ts_tablet_manager.cc:1428] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:57:04.690061 20195 ts_tablet_manager.cc:1428] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260621 07:57:04.691114 20194 tablet_bootstrap.cc:492] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c: Bootstrap starting.
I20260621 07:57:04.691834 20194 tablet_bootstrap.cc:654] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:04.692975 20194 tablet_bootstrap.cc:492] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c: No bootstrap required, opened a new log
I20260621 07:57:04.693063 20194 ts_tablet_manager.cc:1397] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:04.693385 20194 raft_consensus.cc:348] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.693471 20194 raft_consensus.cc:374] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:04.693493 20194 raft_consensus.cc:729] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6670959e07544418990c6ab6b835ba8c, State: Initialized, Role: FOLLOWER
I20260621 07:57:04.693627 20194 consensus_queue.cc:260] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.693856 20194 ts_tablet_manager.cc:1428] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260621 07:57:04.802690 20246 raft_consensus.cc:482] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:04.802807 20246 raft_consensus.cc:504] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.803275 20246 leader_election.cc:288] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:04.803661 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec19a6dacb834d99b5170ac0b781c220" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c" is_pre_election: true
I20260621 07:57:04.803728 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec19a6dacb834d99b5170ac0b781c220" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b" is_pre_election: true
I20260621 07:57:04.803820 19840 raft_consensus.cc:2454] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 0.
I20260621 07:57:04.803913 20120 raft_consensus.cc:2454] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 0.
I20260621 07:57:04.804150 19914 leader_election.cc:302] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6670959e07544418990c6ab6b835ba8c; no voters:
I20260621 07:57:04.804359 20246 raft_consensus.cc:2781] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:04.804421 20246 raft_consensus.cc:482] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:04.804455 20246 raft_consensus.cc:3037] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.805560 20246 raft_consensus.cc:504] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.805866 20246 leader_election.cc:288] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 election: Requested vote from peers 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911), 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277)
I20260621 07:57:04.806160 19840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec19a6dacb834d99b5170ac0b781c220" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6670959e07544418990c6ab6b835ba8c"
I20260621 07:57:04.806180 20120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec19a6dacb834d99b5170ac0b781c220" candidate_uuid: "12ed09a4ecea46448a2468c7b95c375b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e79c78bee5d494bbdb9925fc808203b"
I20260621 07:57:04.806284 19840 raft_consensus.cc:3037] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.806321 20120 raft_consensus.cc:3037] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:04.807219 20120 raft_consensus.cc:2454] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 1.
I20260621 07:57:04.807307 19840 raft_consensus.cc:2454] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ed09a4ecea46448a2468c7b95c375b in term 1.
I20260621 07:57:04.807485 19913 leader_election.cc:302] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ed09a4ecea46448a2468c7b95c375b, 6e79c78bee5d494bbdb9925fc808203b; no voters:
I20260621 07:57:04.807638 20246 raft_consensus.cc:2781] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:04.807744 20246 raft_consensus.cc:686] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 1 LEADER]: Becoming Leader. State: Replica: 12ed09a4ecea46448a2468c7b95c375b, State: Running, Role: LEADER
I20260621 07:57:04.807886 20246 consensus_queue.cc:237] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } }
I20260621 07:57:04.809702 19683 catalog_manager.cc:5495] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b reported cstate change: term changed from 0 to 1, leader changed from <none> to 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130). New cstate: current_term: 1 leader_uuid: "12ed09a4ecea46448a2468c7b95c375b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ed09a4ecea46448a2468c7b95c375b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 } health_report { overall_health: UNKNOWN } } }
I20260621 07:57:04.856359 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.853340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:41205 (local address 127.11.223.130:33855)
0621 07:57:04.853441 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:04.853446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:04.853457 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:04.853580 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:04.853596 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:04.853629 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:04.853722 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:04.853992 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.854516 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.855319 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.855588 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.855693 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.855910 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.855976 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.856045 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:04.856138 (+ 93us) server_negotiation.cc:299] Negotiation successful
0621 07:57:04.856192 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":44}
I20260621 07:57:04.856349 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.853260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33855 (local address 127.11.223.130:41205)
0621 07:57:04.853385 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.853401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:04.853480 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:04.853727 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:04.853729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:04.853735 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0621 07:57:04.853878 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.853883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.854632 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.854637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.855207 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:04.855213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:04.855687 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:04.855691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:04.855775 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:04.855801 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0621 07:57:04.856058 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0621 07:57:04.856065 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 07:57:04.856131 (+ 66us) client_negotiation.cc:240] Negotiation successful
0621 07:57:04.856220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":17}
I20260621 07:57:04.860813 20120 raft_consensus.cc:1261] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Refusing update from remote peer 12ed09a4ecea46448a2468c7b95c375b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.861208 19840 raft_consensus.cc:1261] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Refusing update from remote peer 12ed09a4ecea46448a2468c7b95c375b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:04.861245 20246 consensus_queue.cc:1035] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e79c78bee5d494bbdb9925fc808203b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.861742 20246 consensus_queue.cc:1035] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6670959e07544418990c6ab6b835ba8c" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:04.890022 19683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.890152 19683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:04.891770 19683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a72476c7ec5e4918b60ed604829bad33: Sending DeleteTablet for 3 replicas of tablet ec19a6dacb834d99b5170ac0b781c220
I20260621 07:57:04.892189 19820 tablet_service.cc:1514] Processing DeleteTablet for tablet ec19a6dacb834d99b5170ac0b781c220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:43896
I20260621 07:57:04.892189 19960 tablet_service.cc:1514] Processing DeleteTablet for tablet ec19a6dacb834d99b5170ac0b781c220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260621 07:57:04.892251 20100 tablet_service.cc:1514] Processing DeleteTablet for tablet ec19a6dacb834d99b5170ac0b781c220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:04 UTC) from {username='oryx', principal='oryx/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:59736
I20260621 07:57:04.892436 20219 tablet_replica.cc:330] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: stopping tablet replica
I20260621 07:57:04.892443 20220 tablet_replica.cc:330] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: stopping tablet replica
I20260621 07:57:04.892454 20218 tablet_replica.cc:330] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c: stopping tablet replica
I20260621 07:57:04.892536 20220 raft_consensus.cc:2229] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:57:04.892536 20219 raft_consensus.cc:2229] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.892560 20218 raft_consensus.cc:2229] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:04.892634 20219 raft_consensus.cc:2258] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.892663 20218 raft_consensus.cc:2258] T ec19a6dacb834d99b5170ac0b781c220 P 6670959e07544418990c6ab6b835ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.892689 20220 raft_consensus.cc:2258] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:04.892768 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 19755
I20260621 07:57:04.893652 20219 ts_tablet_manager.cc:1905] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.894873 20220 ts_tablet_manager.cc:1905] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:04.896222 20219 ts_tablet_manager.cc:1918] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:57:04.896337 20219 log.cc:1196] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/ec19a6dacb834d99b5170ac0b781c220
I20260621 07:57:04.896683 20219 ts_tablet_manager.cc:1939] T ec19a6dacb834d99b5170ac0b781c220 P 6e79c78bee5d494bbdb9925fc808203b: Deleting consensus metadata
I20260621 07:57:04.897274 19669 catalog_manager.cc:4841] TS 6e79c78bee5d494bbdb9925fc808203b (127.11.223.131:38277): tablet ec19a6dacb834d99b5170ac0b781c220 (table test-table [id=c92df2f190554ee9bc72bfd35f7b6661]) successfully deleted
I20260621 07:57:04.897418 20220 ts_tablet_manager.cc:1918] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260621 07:57:04.897495 20220 log.cc:1196] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/ec19a6dacb834d99b5170ac0b781c220
I20260621 07:57:04.897778 20220 ts_tablet_manager.cc:1939] T ec19a6dacb834d99b5170ac0b781c220 P 12ed09a4ecea46448a2468c7b95c375b: Deleting consensus metadata
I20260621 07:57:04.898308 19669 catalog_manager.cc:4841] TS 12ed09a4ecea46448a2468c7b95c375b (127.11.223.130:33855): tablet ec19a6dacb834d99b5170ac0b781c220 (table test-table [id=c92df2f190554ee9bc72bfd35f7b6661]) successfully deleted
W20260621 07:57:04.901957 19670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:36911 (error 108)
I20260621 07:57:04.902175 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 19895
I20260621 07:57:04.904436 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:04.904049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36911 (local address 127.0.0.1:43912)
0621 07:57:04.904251 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:04.904310 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:36911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":62,"threads_started":1}
W20260621 07:57:04.904554 19670 catalog_manager.cc:4584] TS 6670959e07544418990c6ab6b835ba8c (127.11.223.129:36911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec19a6dacb834d99b5170ac0b781c220: Network error: Client connection negotiation failed: client connection to 127.11.223.129:36911: connect: Connection refused (error 111)
I20260621 07:57:04.909348 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20035
I20260621 07:57:04.915555 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 19651
2026-06-21T07:57:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3205 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[20288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[20288](info): set up 2 sockets
Jun 21 07:57:04 dist-test-slave-0pn3 krb5kdc[20288](info): commencing operation
krb5kdc: starting...
W20260621 07:57:05.917047 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:57:05 dist-test-slave-0pn3 krb5kdc[20288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:57:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:57:06.028110 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45631
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:06.129912 20301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:06.130162 20301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:06.130244 20301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:06.134305 20301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:06.134382 20301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:06.134416 20301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:06.134451 20301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:06.134493 20301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:06.139117 20301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:06.139976 20301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:06.140947 20301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:06.148614 20307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:06.148605 20306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:06.148639 20301 server_base.cc:1034] running on GCE node
W20260621 07:57:06.148612 20309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:06.149324 20301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:06.150400 20301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:06.151598 20301 hybrid_clock.cc:648] HybridClock initialized: now 1782028626151584 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-21T07:57:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1239 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:57:06 dist-test-slave-0pn3 krb5kdc[20314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:57:06 dist-test-slave-0pn3 krb5kdc[20314](info): set up 2 sockets
Jun 21 07:57:06 dist-test-slave-0pn3 krb5kdc[20314](info): commencing operation
krb5kdc: starting...
W20260621 07:57:07.173642 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:57:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:57:07.283989 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45057
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:07.386886 20327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:07.387149 20327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:07.387212 20327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:07.391099 20327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:07.391178 20327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:07.391217 20327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:07.391273 20327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:07.391314 20327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:07.396154 20327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:07.397092 20327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:07.397974 20327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:07.405720 20335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:07.405735 20332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:07.405730 20333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:07.405946 20327 server_base.cc:1034] running on GCE node
I20260621 07:57:07.406456 20327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:07.407424 20327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:07.408600 20327 hybrid_clock.cc:648] HybridClock initialized: now 1782028627408580 us; error 42 us; skew 500 ppm
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:07.411125 20327 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:57:07.411945 20327 webserver.cc:460] Webserver started at http://127.11.223.190:43959/ using document root <none> and password file <none>
I20260621 07:57:07.412516 20327 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:07.412602 20327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:07.412806 20327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:07.414661 20327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "ba3a93656a3d414b9da7c06992110a3f"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "ffc04781e4a3927666a0b3a9564bc969"
server_key_iv: "3a46c06601ec996d291846e03f04b22a"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.415220 20327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "ba3a93656a3d414b9da7c06992110a3f"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "ffc04781e4a3927666a0b3a9564bc969"
server_key_iv: "3a46c06601ec996d291846e03f04b22a"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.418699 20327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:57:07.420920 20342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.422036 20327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:07.422163 20327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ba3a93656a3d414b9da7c06992110a3f"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "ffc04781e4a3927666a0b3a9564bc969"
server_key_iv: "3a46c06601ec996d291846e03f04b22a"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.422263 20327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:07.439685 20327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:07.442345 20327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:07.442538 20327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:07.449277 20327 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:45057
I20260621 07:57:07.449304 20394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:45057 every 8 connection(s)
I20260621 07:57:07.450213 20327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:57:07.452771 20395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:07.457806 20395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Bootstrap starting.
I20260621 07:57:07.459393 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20327
I20260621 07:57:07.459529 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:57:07.459821 12158 external_mini_cluster.cc:1428] Setting key d5ea6dabce89b85c4c8a99837c61e343
I20260621 07:57:07.460510 20395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:07.461385 20395 log.cc:824] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:07.463245 20395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: No bootstrap required, opened a new log
I20260621 07:57:07.466619 20395 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:07.466898 20395 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:07.466969 20395 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3a93656a3d414b9da7c06992110a3f, State: Initialized, Role: FOLLOWER
I20260621 07:57:07.467429 20395 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:07.467554 20395 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:57:07.467634 20395 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:57:07.467723 20395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 0 FOLLOWER]: Advancing to term 1
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:07.468891 20395 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:07.469245 20395 leader_election.cc:302] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba3a93656a3d414b9da7c06992110a3f; no voters:
I20260621 07:57:07.469640 20395 leader_election.cc:288] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:57:07.469746 20400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:07.470026 20400 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 LEADER]: Becoming Leader. State: Replica: ba3a93656a3d414b9da7c06992110a3f, State: Running, Role: LEADER
I20260621 07:57:07.470422 20400 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:07.470927 20395 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:57:07.471647 20401 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba3a93656a3d414b9da7c06992110a3f. Latest consensus state: current_term: 1 leader_uuid: "ba3a93656a3d414b9da7c06992110a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } } }
I20260621 07:57:07.471760 20401 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:07.472496 20402 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba3a93656a3d414b9da7c06992110a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } } }
I20260621 07:57:07.472675 20402 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:07.472992 20406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:07.475020 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:07.460992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.11.223.190:45057)
0621 07:57:07.461426 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:07.461441 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:07.461475 (+ 34us) server_negotiation.cc:407] Connection header received
0621 07:57:07.462356 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:07.462375 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:07.462687 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:07.463032 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:07.463827 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.464845 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.465703 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.466150 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.466315 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:07.469187 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:07.469192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:07.469204 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:07.469229 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:07.471447 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:07.472016 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:07.472037 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:07.472043 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:07.472186 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:07.472563 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:07.472570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:07.472575 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:07.473160 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:07.473371 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:07.474100 (+ 729us) server_negotiation.cc:299] Negotiation successful
0621 07:57:07.474350 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":133,"threads_started":1}
I20260621 07:57:07.476612 20406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:07.482002 20406 catalog_manager.cc:1347] Generated new cluster ID: 7e890fc3d1d142cb8659ce98b2e993e9
I20260621 07:57:07.482074 20406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:07.493615 20406 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:57:07.494722 20406 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:07.500905 20406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Generated new TSK 0
I20260621 07:57:07.501605 20406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:57:07.533257 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45057
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:07.653973 20421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:07.654218 20421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:07.654258 20421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:07.658000 20421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:07.658053 20421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:07.658108 20421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:57:07.663033 20421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:07.663961 20421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:07.665407 20421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:07.674814 20426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:07.674939 20421 server_base.cc:1034] running on GCE node
W20260621 07:57:07.674832 20427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:07.674832 20429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:07.675437 20421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:07.676038 20421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:07.677196 20421 hybrid_clock.cc:648] HybridClock initialized: now 1782028627677162 us; error 45 us; skew 500 ppm
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:07.679715 20421 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:57:07.680567 20421 webserver.cc:460] Webserver started at http://127.11.223.129:33839/ using document root <none> and password file <none>
I20260621 07:57:07.681088 20421 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:07.681149 20421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:07.681391 20421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:07.683261 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "af4234ae894a441e9b71b0278cd691e7"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "4e3325372d5a7b7a39382e9c767204d5"
server_key_iv: "c1c0b717884fa1cab0b516c09b7886de"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.683828 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "af4234ae894a441e9b71b0278cd691e7"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "4e3325372d5a7b7a39382e9c767204d5"
server_key_iv: "c1c0b717884fa1cab0b516c09b7886de"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.687165 20421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260621 07:57:07.689482 20436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.690663 20421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:07.690794 20421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "af4234ae894a441e9b71b0278cd691e7"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "4e3325372d5a7b7a39382e9c767204d5"
server_key_iv: "c1c0b717884fa1cab0b516c09b7886de"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.690898 20421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:07.708902 20421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:07.711587 20421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:07.711784 20421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:07.712404 20421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:07.713222 20421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:07.713289 20421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.713358 20421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:07.713400 20421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.721979 20421 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:43255
I20260621 07:57:07.722007 20549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:43255 every 8 connection(s)
I20260621 07:57:07.722803 20421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:57:07.729431 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20421
I20260621 07:57:07.729590 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:57:07.729919 12158 external_mini_cluster.cc:1428] Setting key 64190f1d07705150131204b65c582eff
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:07.737210 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:07.724566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:51091 (local address 127.11.223.190:45057)
0621 07:57:07.724697 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:07.724701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:07.725464 (+ 763us) server_negotiation.cc:407] Connection header received
0621 07:57:07.726738 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:07.726741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:07.726787 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:07.726857 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:07.727901 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.728647 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.729892 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.730237 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.730359 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:07.733176 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:07.733183 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:07.733187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:07.733220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:07.734863 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:07.735334 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:07.735340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:07.735344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:07.735454 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:07.735827 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:07.735834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:07.735837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:07.736083 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:07.736191 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:07.736834 (+ 643us) server_negotiation.cc:299] Negotiation successful
0621 07:57:07.736978 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 07:57:07.737819 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:07.724925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:51091)
0621 07:57:07.725330 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:07.725370 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:07.726554 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:07.727006 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:07.727020 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:07.727341 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:07.727774 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:07.727785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.728789 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.728792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:07.729775 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:07.729784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.730401 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.730405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:07.730595 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:07.731170 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:07.731185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:07.732983 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:07.734972 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:07.734977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:07.734987 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:07.735240 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:07.735548 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:07.735550 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:07.735551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:07.735715 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:07.736279 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:07.736284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:07.736586 (+ 302us) client_negotiation.cc:769] Sending connection context
0621 07:57:07.736746 (+ 160us) client_negotiation.cc:240] Negotiation successful
0621 07:57:07.736979 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":113,"threads_started":1}
I20260621 07:57:07.738967 20550 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45057
I20260621 07:57:07.739207 20550 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:57:07.739656 20550 heartbeater.cc:511] Master 127.11.223.190:45057 requested a full tablet report, sending...
I20260621 07:57:07.741287 20359 ts_manager.cc:194] Registered new tserver with Master: af4234ae894a441e9b71b0278cd691e7 (127.11.223.129:43255)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:57:07.742751 20359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:51091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:57:07.755225 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45057
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:07.861801 20555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:07.862035 20555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:07.862074 20555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:07.865787 20555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:07.865831 20555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:07.865883 20555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:57:07.870906 20555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:07.871992 20555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:07.873098 20555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:07.881958 20561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:07.881958 20560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:07.881978 20563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:07.882014 20555 server_base.cc:1034] running on GCE node
I20260621 07:57:07.882544 20555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:07.883101 20555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:07.884277 20555 hybrid_clock.cc:648] HybridClock initialized: now 1782028627884270 us; error 29 us; skew 500 ppm
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:07.886694 20555 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:57:07.887476 20555 webserver.cc:460] Webserver started at http://127.11.223.130:36831/ using document root <none> and password file <none>
I20260621 07:57:07.887988 20555 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:07.888094 20555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:07.888302 20555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:07.890221 20555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "353852f6b3874f35b53c8eac8ba4c4de"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "8958943501e612ac674a949680da596c"
server_key_iv: "bb2ac8605645c8432780f7eb256dffef"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.890774 20555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "353852f6b3874f35b53c8eac8ba4c4de"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "8958943501e612ac674a949680da596c"
server_key_iv: "bb2ac8605645c8432780f7eb256dffef"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.894130 20555 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260621 07:57:07.896271 20570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.897397 20555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:07.897521 20555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "353852f6b3874f35b53c8eac8ba4c4de"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "8958943501e612ac674a949680da596c"
server_key_iv: "bb2ac8605645c8432780f7eb256dffef"
server_key_version: "encryptionkey@0"
I20260621 07:57:07.897617 20555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:07.918560 20555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:07.921255 20555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:07.921440 20555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:07.921967 20555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:07.922698 20555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:07.922765 20555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.922832 20555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:07.922879 20555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:07.931552 20555 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:33369
I20260621 07:57:07.931578 20683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:33369 every 8 connection(s)
I20260621 07:57:07.932415 20555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:57:07.940591 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20555
I20260621 07:57:07.940687 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
Jun 21 07:57:07 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:07.941022 12158 external_mini_cluster.cc:1428] Setting key a372be1f2bcc38864d60bebcaaf07346
I20260621 07:57:07.945323 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:07.934003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:49083 (local address 127.11.223.190:45057)
0621 07:57:07.934158 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:07.934162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:07.934922 (+ 760us) server_negotiation.cc:407] Connection header received
0621 07:57:07.936078 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:07.936081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:07.936125 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:07.936195 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:07.937282 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.937780 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.938792 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.938989 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.939061 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:07.941504 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:07.941519 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:07.941521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:07.941548 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:07.943233 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:07.943744 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:07.943750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:07.943755 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:07.943845 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:07.944185 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:07.944189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:07.944190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:07.944454 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:07.944559 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:07.945135 (+ 576us) server_negotiation.cc:299] Negotiation successful
0621 07:57:07.945218 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 07:57:07.946437 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:07.934379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:49083)
0621 07:57:07.934782 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:07.934820 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:07.935899 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:07.936387 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:07.936395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:07.936739 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:07.937158 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:07.937169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.937878 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.937880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:07.938688 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:07.938696 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:07.939105 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:07.939108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:07.939231 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:07.939772 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:07.939787 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:07.941346 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:07.943359 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:07.943363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:07.943373 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:07.943647 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:07.943935 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:07.943937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:07.943938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:07.944091 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:07.944575 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:07.944582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:07.944895 (+ 313us) client_negotiation.cc:769] Sending connection context
0621 07:57:07.945100 (+ 205us) client_negotiation.cc:240] Negotiation successful
0621 07:57:07.945409 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":111,"threads_started":1}
I20260621 07:57:07.947567 20684 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45057
I20260621 07:57:07.947788 20684 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:07.948215 20684 heartbeater.cc:511] Master 127.11.223.190:45057 requested a full tablet report, sending...
I20260621 07:57:07.949211 20359 ts_manager.cc:194] Registered new tserver with Master: 353852f6b3874f35b53c8eac8ba4c4de (127.11.223.130:33369)
I20260621 07:57:07.949779 20359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:49083
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:57:07.965382 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45057
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:08.067377 20689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:08.067651 20689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:08.067721 20689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:08.071568 20689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:08.071683 20689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:08.071768 20689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:57:08.076618 20689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:08.077457 20689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:08.078486 20689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:08.087342 20695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.087352 20694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.087345 20697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.087530 20689 server_base.cc:1034] running on GCE node
I20260621 07:57:08.087878 20689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:08.088490 20689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:08.089645 20689 hybrid_clock.cc:648] HybridClock initialized: now 1782028628089622 us; error 41 us; skew 500 ppm
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:08.092267 20689 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:57:08.093024 20689 webserver.cc:460] Webserver started at http://127.11.223.131:43495/ using document root <none> and password file <none>
I20260621 07:57:08.093546 20689 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:08.093632 20689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:08.093842 20689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:08.095739 20689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "751a21db71204803a494948c6a39de05"
format_stamp: "Formatted at 2026-06-21 07:57:08 on dist-test-slave-0pn3"
server_key: "3bdcd00cee3e33fa92bd0fe03029234a"
server_key_iv: "ccf0b5038511df1e67584c39531c449b"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.096196 20689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "751a21db71204803a494948c6a39de05"
format_stamp: "Formatted at 2026-06-21 07:57:08 on dist-test-slave-0pn3"
server_key: "3bdcd00cee3e33fa92bd0fe03029234a"
server_key_iv: "ccf0b5038511df1e67584c39531c449b"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.099326 20689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260621 07:57:08.101671 20704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.102766 20689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:08.102898 20689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "751a21db71204803a494948c6a39de05"
format_stamp: "Formatted at 2026-06-21 07:57:08 on dist-test-slave-0pn3"
server_key: "3bdcd00cee3e33fa92bd0fe03029234a"
server_key_iv: "ccf0b5038511df1e67584c39531c449b"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.102998 20689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:08.119066 20689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:08.121850 20689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:08.122030 20689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:08.122527 20689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:08.123260 20689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:08.123327 20689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.123389 20689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:08.123440 20689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.131672 20689 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:46853
I20260621 07:57:08.131692 20817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:46853 every 8 connection(s)
I20260621 07:57:08.132540 20689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:57:08.140542 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20689
I20260621 07:57:08.140692 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:57:08.140965 12158 external_mini_cluster.cc:1428] Setting key 11f6fa26c41419d0b89725ca1a030960
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:08.145833 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.134297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:46095 (local address 127.11.223.190:45057)
0621 07:57:08.134424 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.134428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.135204 (+ 776us) server_negotiation.cc:407] Connection header received
0621 07:57:08.136529 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.136532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.136593 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.136669 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.137721 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.138238 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.139322 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.139547 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.139655 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.142312 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:08.142316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:08.142319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:08.142349 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:08.143760 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:08.144284 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:08.144288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:08.144290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:08.144400 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:08.144745 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:08.144749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:08.144751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:08.144933 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:08.145038 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:08.145555 (+ 517us) server_negotiation.cc:299] Negotiation successful
0621 07:57:08.145669 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:57:08.146570 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.134662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:46095)
0621 07:57:08.135073 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.135112 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.136246 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.136825 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.136831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.137165 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.137582 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.137593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.138364 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.138372 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.139192 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.139200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.139690 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.139693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.139850 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.140498 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.140513 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.142157 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:08.143901 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:08.143907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:08.143921 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:08.144179 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:08.144509 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:08.144511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:08.144513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:08.144648 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:08.145055 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:08.145079 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:08.145319 (+ 240us) client_negotiation.cc:769] Sending connection context
0621 07:57:08.145495 (+ 176us) client_negotiation.cc:240] Negotiation successful
0621 07:57:08.145797 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":113,"threads_started":1}
I20260621 07:57:08.147549 20818 heartbeater.cc:347] Connected to a master server at 127.11.223.190:45057
I20260621 07:57:08.147776 20818 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:08.148223 20818 heartbeater.cc:511] Master 127.11.223.190:45057 requested a full tablet report, sending...
I20260621 07:57:08.149170 20359 ts_manager.cc:194] Registered new tserver with Master: 751a21db71204803a494948c6a39de05 (127.11.223.131:46853)
I20260621 07:57:08.149734 20359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:46095
I20260621 07:57:08.154812 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab.
Entry for principal oryx/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab.
WARNING: no policy specified for oryx/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab.
Entry for principal oryx/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab.
WARNING: no policy specified for oryx/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab.
Entry for principal oryx/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab.
I20260621 07:57:08.218962 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20421
I20260621 07:57:08.224104 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20555
I20260621 07:57:08.229204 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20689
I20260621 07:57:08.234362 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20327
I20260621 07:57:08.240175 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45057
--webserver_interface=127.11.223.190
--webserver_port=43959
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:08.345367 20827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:08.345649 20827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:08.345724 20827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:08.349669 20827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:08.349748 20827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:08.349786 20827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:08.349844 20827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:08.349884 20827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:08.355078 20827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=43959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:08.356087 20827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:08.357167 20827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:08.365247 20832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.365280 20827 server_base.cc:1034] running on GCE node
W20260621 07:57:08.365238 20835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.365238 20833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.365906 20827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:08.366962 20827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:08.368124 20827 hybrid_clock.cc:648] HybridClock initialized: now 1782028628368110 us; error 43 us; skew 500 ppm
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:08.370818 20827 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:57:08.371703 20827 webserver.cc:460] Webserver started at http://127.11.223.190:43959/ using document root <none> and password file <none>
I20260621 07:57:08.372234 20827 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:08.372373 20827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:08.375929 20827 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260621 07:57:08.378010 20842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.379168 20827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:08.379303 20827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ba3a93656a3d414b9da7c06992110a3f"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "ffc04781e4a3927666a0b3a9564bc969"
server_key_iv: "3a46c06601ec996d291846e03f04b22a"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.379822 20827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:08.399962 20827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:08.402572 20827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:08.402753 20827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:08.409281 20827 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:45057
I20260621 07:57:08.409293 20894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:45057 every 8 connection(s)
I20260621 07:57:08.410300 20827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:57:08.414304 20895 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:57:08.415787 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20827
I20260621 07:57:08.415992 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Bootstrap starting.
I20260621 07:57:08.416301 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:43255
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:45057
--webserver_port=33839
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:57:08.432250 20895 log.cc:824] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:08.438493 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:57:08.438851 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Bootstrap complete.
I20260621 07:57:08.442963 20895 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:08.443338 20895 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3a93656a3d414b9da7c06992110a3f, State: Initialized, Role: FOLLOWER
I20260621 07:57:08.443795 20895 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:08.443898 20895 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:57:08.443984 20895 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:57:08.444078 20895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:57:08.445856 20895 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:08.446166 20895 leader_election.cc:302] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba3a93656a3d414b9da7c06992110a3f; no voters:
I20260621 07:57:08.446463 20895 leader_election.cc:288] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:57:08.446539 20899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:57:08.446786 20899 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [term 2 LEADER]: Becoming Leader. State: Replica: ba3a93656a3d414b9da7c06992110a3f, State: Running, Role: LEADER
I20260621 07:57:08.447118 20899 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } }
I20260621 07:57:08.447636 20895 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:57:08.447956 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba3a93656a3d414b9da7c06992110a3f. Latest consensus state: current_term: 2 leader_uuid: "ba3a93656a3d414b9da7c06992110a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } } }
I20260621 07:57:08.448365 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:08.448719 20900 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ba3a93656a3d414b9da7c06992110a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3a93656a3d414b9da7c06992110a3f" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45057 } } }
I20260621 07:57:08.448896 20900 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:08.449633 20908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:08.452633 20908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:08.453248 20908 catalog_manager.cc:1259] Loaded cluster ID: 7e890fc3d1d142cb8659ce98b2e993e9
I20260621 07:57:08.453310 20908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:08.455497 20908 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:08.456118 20908 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba3a93656a3d414b9da7c06992110a3f: Loaded TSK: 0
I20260621 07:57:08.457275 20908 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 07:57:08.523337 20897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:08.523581 20897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:08.523648 20897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:08.527441 20897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:08.527518 20897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:08.527596 20897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:57:08.532685 20897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.129.keytab
--principal=oryx/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:43255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=33839
--tserver_master_addrs=127.11.223.190:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:08.533545 20897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:08.534437 20897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:08.543036 20921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.543042 20922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.543048 20924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.543471 20897 server_base.cc:1034] running on GCE node
I20260621 07:57:08.543869 20897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:08.544600 20897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:08.545795 20897 hybrid_clock.cc:648] HybridClock initialized: now 1782028628545746 us; error 59 us; skew 500 ppm
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028628, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:08.548955 20897 init.cc:377] Logged in from keytab as oryx/127.11.223.129@KRBTEST.COM (short username oryx)
I20260621 07:57:08.549890 20897 webserver.cc:460] Webserver started at http://127.11.223.129:33839/ using document root <none> and password file <none>
I20260621 07:57:08.550413 20897 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:08.550485 20897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:08.554162 20897 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:57:08.556100 20931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.557292 20897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:08.557428 20897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "af4234ae894a441e9b71b0278cd691e7"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "4e3325372d5a7b7a39382e9c767204d5"
server_key_iv: "c1c0b717884fa1cab0b516c09b7886de"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.557925 20897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:08.594504 20897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:08.597584 20897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:08.597790 20897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:08.598405 20897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:08.599171 20897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:08.599241 20897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.599309 20897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:08.599380 20897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.607620 20897 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:43255
I20260621 07:57:08.607647 21044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:43255 every 8 connection(s)
I20260621 07:57:08.608541 20897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:57:08.612587 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 20897
I20260621 07:57:08.613032 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:33369
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:45057
--webserver_port=36831
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.622242 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.610498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:40633)
0621 07:57:08.610943 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.610991 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.612140 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.613229 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.613240 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.613719 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.614380 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.614401 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.616302 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.616305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.617294 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.617304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.618396 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.618400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.618527 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.619083 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.619097 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.621600 (+ 2503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":151,"threads_started":1}
W20260621 07:57:08.622716 21045 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:45057 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.223.190:45057: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
W20260621 07:57:08.622785 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.610400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:40633 (local address 127.11.223.190:45057)
0621 07:57:08.610815 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.610833 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.611099 (+ 266us) server_negotiation.cc:407] Connection header received
0621 07:57:08.612492 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.612503 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.612803 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.613068 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.614588 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.616148 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.617426 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.618241 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.618430 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.621884 (+ 3454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:40633: BlockingRecv error: recv got EOF from 127.11.223.129:40633 (error 108)
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.628182 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.623100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:39211)
0621 07:57:08.623247 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.623261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.623332 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.623677 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.623679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.623798 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.623880 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.623884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.624975 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.624977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.625479 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.625485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.625995 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.625997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.626058 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.626472 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.626482 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.628105 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260621 07:57:08.628371 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.623212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:39211 (local address 127.11.223.190:45057)
0621 07:57:08.623394 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.623400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.623417 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:57:08.623456 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.623462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.623529 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.623663 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.623993 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.624882 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.625578 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.625898 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.625991 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.628244 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:39211: BlockingRecv error: recv got EOF from 127.11.223.129:39211 (error 108)
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":103}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.633603 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.628870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:38913)
0621 07:57:08.628952 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.628968 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.629026 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.629228 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.629230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.629337 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.629429 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.629433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.630368 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.630370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.630872 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.630878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.631419 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.631421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.631473 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.631897 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.631907 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.633532 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260621 07:57:08.633759 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.628927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:38913 (local address 127.11.223.190:45057)
0621 07:57:08.629013 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.629017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.629029 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:57:08.629098 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.629100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.629135 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.629210 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.629529 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.630266 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.630982 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.631311 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.631409 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.633668 (+ 2259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:38913: BlockingRecv error: recv got EOF from 127.11.223.129:38913 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260621 07:57:08.633775 21045 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:57:08.720592 21049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:08.720801 21049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:08.720839 21049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:08.724570 21049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:08.724613 21049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:08.724665 21049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:57:08.729306 21049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.130.keytab
--principal=oryx/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:33369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=36831
--tserver_master_addrs=127.11.223.190:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:08.730155 21049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:08.731010 21049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:08.739611 21057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.739641 21049 server_base.cc:1034] running on GCE node
W20260621 07:57:08.739611 21055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.739804 21054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.740272 21049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:08.740888 21049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:08.742069 21049 hybrid_clock.cc:648] HybridClock initialized: now 1782028628742059 us; error 26 us; skew 500 ppm
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028628, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:08.744645 21049 init.cc:377] Logged in from keytab as oryx/127.11.223.130@KRBTEST.COM (short username oryx)
I20260621 07:57:08.745388 21049 webserver.cc:460] Webserver started at http://127.11.223.130:36831/ using document root <none> and password file <none>
I20260621 07:57:08.745870 21049 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:08.745970 21049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:08.749783 21049 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:57:08.751647 21064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.752902 21049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:08.753028 21049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "353852f6b3874f35b53c8eac8ba4c4de"
format_stamp: "Formatted at 2026-06-21 07:57:07 on dist-test-slave-0pn3"
server_key: "8958943501e612ac674a949680da596c"
server_key_iv: "bb2ac8605645c8432780f7eb256dffef"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.753525 21049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:08.771234 21049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:08.774279 21049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:08.774482 21049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:08.775019 21049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:08.775736 21049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:08.775803 21049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.775866 21049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:08.775920 21049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.784435 21049 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:33369
I20260621 07:57:08.784463 21177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:33369 every 8 connection(s)
I20260621 07:57:08.785281 21049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:57:08.789274 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21049
I20260621 07:57:08.789839 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:46853
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:45057
--webserver_port=43495
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
W20260621 07:57:08.795755 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.786880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:48321 (local address 127.11.223.190:45057)
0621 07:57:08.787045 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.787050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.787771 (+ 721us) server_negotiation.cc:407] Connection header received
0621 07:57:08.788924 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.788926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.788974 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.789052 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.790573 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.791159 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.792240 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.792508 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.792615 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.795643 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:48321: BlockingRecv error: recv got EOF from 127.11.223.130:48321 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:57:08.796252 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.787237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:48321)
0621 07:57:08.787632 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.787671 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.788747 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.789260 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.789269 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.789766 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.790421 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.790443 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.791287 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.791290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.792127 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.792136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.792644 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.792648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.792798 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.793297 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.793311 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.795544 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":107,"threads_started":1}
W20260621 07:57:08.796677 21178 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:45057 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.223.190:45057: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.801672 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.797062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:49319)
0621 07:57:08.797151 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.797166 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.797235 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.797437 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.797440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.797578 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.797652 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.797657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.798361 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.798368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.798891 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.798898 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.799338 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.799341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.799420 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.799863 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.799874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.801538 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260621 07:57:08.801883 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.797125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:49319 (local address 127.11.223.190:45057)
0621 07:57:08.797226 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.797229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.797241 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:57:08.797307 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.797311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.797349 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.797425 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.797780 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.798259 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.799058 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.799241 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.799337 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.801661 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:49319: BlockingRecv error: recv got EOF from 127.11.223.130:49319 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.806972 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.802232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:59875)
0621 07:57:08.802359 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.802370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.802441 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.802638 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.802641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.802786 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.802859 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.802864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.803473 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.803475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.804313 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.804323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.804723 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.804726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.804819 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.805170 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.805182 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.806870 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260621 07:57:08.807121 21178 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:57:08.807268 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.802296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:59875 (local address 127.11.223.190:45057)
0621 07:57:08.802401 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.802404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.802415 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:08.802513 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.802515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.802544 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.802604 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.802955 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.803383 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.804445 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.804629 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.804696 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.806968 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:59875: BlockingRecv error: recv got EOF from 127.11.223.130:59875 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260621 07:57:08.894737 21181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:08.895057 21181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:08.895143 21181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:08.898942 21181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:08.899030 21181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:08.899116 21181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:57:08.903900 21181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/krb5kdc/oryx_127.11.223.131.keytab
--principal=oryx/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:46853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=43495
--tserver_master_addrs=127.11.223.190:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:08.904889 21181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:08.905802 21181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:08.914292 21189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.914366 21187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:08.914443 21186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:08.915071 21181 server_base.cc:1034] running on GCE node
I20260621 07:57:08.915458 21181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:08.916018 21181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:08.917183 21181 hybrid_clock.cc:648] HybridClock initialized: now 1782028628917173 us; error 26 us; skew 500 ppm
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028628, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:08.919685 21181 init.cc:377] Logged in from keytab as oryx/127.11.223.131@KRBTEST.COM (short username oryx)
I20260621 07:57:08.920774 21181 webserver.cc:460] Webserver started at http://127.11.223.131:43495/ using document root <none> and password file <none>
I20260621 07:57:08.921310 21181 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:08.921412 21181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:08.924744 21181 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:57:08.926630 21196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.927748 21181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:08.927883 21181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "751a21db71204803a494948c6a39de05"
format_stamp: "Formatted at 2026-06-21 07:57:08 on dist-test-slave-0pn3"
server_key: "3bdcd00cee3e33fa92bd0fe03029234a"
server_key_iv: "ccf0b5038511df1e67584c39531c449b"
server_key_version: "encryptionkey@0"
I20260621 07:57:08.928439 21181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:08.948655 21181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:08.951695 21181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:08.951869 21181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:08.952461 21181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:08.953258 21181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:08.953301 21181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.953365 21181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:08.953405 21181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:08.961469 21181 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:46853
I20260621 07:57:08.961499 21309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:46853 every 8 connection(s)
I20260621 07:57:08.962709 21181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:57:08.965883 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21181
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
W20260621 07:57:08.974118 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.964383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:49455 (local address 127.11.223.190:45057)
0621 07:57:08.964506 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.964509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.965256 (+ 747us) server_negotiation.cc:407] Connection header received
0621 07:57:08.966629 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.966635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.966686 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.966797 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.968097 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.968662 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.969887 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.970272 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.970371 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.973971 (+ 3600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:49455: BlockingRecv error: recv got EOF from 127.11.223.131:49455 (error 108)
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":40}
I20260621 07:57:08.974756 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.964699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:49455)
0621 07:57:08.965110 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.965149 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.966405 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.967074 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.967083 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.967501 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.967964 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.967980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.968783 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.968785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.969766 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.969775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.970423 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.970429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.970585 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.971074 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.971089 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.973883 (+ 2794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
W20260621 07:57:08.975231 21310 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:45057 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.223.190:45057: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
I20260621 07:57:08.975802 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.966837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.11.223.190:45057)
0621 07:57:08.967074 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.967078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.967089 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:08.967123 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.967126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.967163 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.967270 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.967939 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.968582 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.969457 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.969776 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.969886 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.971269 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:08.971276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:08.971292 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:08.971322 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:08.973329 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:08.973790 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:08.973796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:08.973801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:08.973940 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:08.974280 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:08.974284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:08.974286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:08.974671 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:08.974860 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:08.975389 (+ 529us) server_negotiation.cc:299] Negotiation successful
0621 07:57:08.975641 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":160,"thread_start_us":79,"threads_started":1}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.980701 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.975603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:50561)
0621 07:57:08.975740 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.975751 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.975815 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.976573 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.976575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.976723 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.976808 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.976813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.977548 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.977551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.978232 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.978240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.978653 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.978656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.978738 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.979077 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.979087 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.980578 (+ 1491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260621 07:57:08.980784 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.975668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:50561 (local address 127.11.223.190:45057)
0621 07:57:08.975776 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.975779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.975790 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:08.976424 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.976429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.976471 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.976559 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.976912 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.977437 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.978345 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.978550 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.978652 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.980681 (+ 2029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:50561: BlockingRecv error: recv got EOF from 127.11.223.131:50561 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:08 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:08.985270 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:08.981152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:55187)
0621 07:57:08.981266 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:08.981295 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:08.981368 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:08.981568 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:08.981571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:08.981711 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:08.981768 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.981772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.982423 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.982429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.982894 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:08.982901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.983316 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.983319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:08.983396 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.983704 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:08.983714 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:08.985161 (+ 1447us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260621 07:57:08.985338 21047 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:08.981242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:55187 (local address 127.11.223.190:45057)
0621 07:57:08.981377 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:08.981381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:08.981395 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:57:08.981432 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:08.981437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:08.981477 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:08.981556 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:08.981881 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.982327 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.983009 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:08.983219 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:08.983315 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:08.985245 (+ 1930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:55187: BlockingRecv error: recv got EOF from 127.11.223.131:55187 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260621 07:57:08.985458 21310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 21 07:57:09 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:09 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:09.639456 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:09.634309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:57683)
0621 07:57:09.634551 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:09.634564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:09.634644 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:09.634855 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:09.634857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:09.635010 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:09.635110 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:09.635115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.636135 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.636138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:09.636757 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:09.636766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.637169 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.637171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:09.637233 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:09.637627 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:09.637637 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:09.639361 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
W20260621 07:57:09.639632 21316 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:09.634379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:57683 (local address 127.11.223.190:45057)
0621 07:57:09.634622 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:09.634625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:09.634637 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:57:09.634719 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:09.634721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:09.634761 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:09.634832 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:09.635319 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.636008 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.636856 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.637083 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.637155 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:09.639475 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:57683: BlockingRecv error: recv got EOF from 127.11.223.129:57683 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":67,"threads_started":1}
Jun 21 07:57:09 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:09 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:09.812677 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:09.807667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:33221)
0621 07:57:09.807924 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:09.807937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:09.808028 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:09.808242 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:09.808244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:09.808423 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:09.808521 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:09.808526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.809193 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.809195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:09.809770 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:09.809777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.810171 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.810173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:09.810232 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:09.810685 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:09.810696 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:09.812569 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
W20260621 07:57:09.812855 21316 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:09.807708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:33221 (local address 127.11.223.190:45057)
0621 07:57:09.807874 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:09.807877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:09.808007 (+ 130us) server_negotiation.cc:407] Connection header received
0621 07:57:09.808103 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:09.808105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:09.808144 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:09.808234 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:09.808614 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.809102 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.809867 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.810074 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.810145 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:09.812700 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:33221: BlockingRecv error: recv got EOF from 127.11.223.130:33221 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Jun 21 07:57:09 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:09 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:09.995206 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:09.986053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:56809)
0621 07:57:09.986288 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:09.986300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:09.986381 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:09.986601 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:09.986603 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:09.986753 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:09.986838 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:09.986843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.987613 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.987616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:09.988377 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:09.988386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.988839 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.988841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:09.988900 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:09.993287 (+ 4387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:09.993298 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:09.995103 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
W20260621 07:57:09.995379 21316 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:09.986094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:56809 (local address 127.11.223.190:45057)
0621 07:57:09.986227 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:09.986230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:09.986367 (+ 137us) server_negotiation.cc:407] Connection header received
0621 07:57:09.986456 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:09.986458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:09.986503 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:09.986568 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:09.986930 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.987515 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.988486 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:09.988744 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:09.988823 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:09.995221 (+ 6398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:56809: BlockingRecv error: recv got EOF from 127.11.223.131:56809 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 21 07:57:10 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:10 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:10.645004 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:10.640133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:52617)
0621 07:57:10.640408 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:10.640421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:10.640510 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:10.640763 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:10.640765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:10.640928 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:10.641019 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:10.641024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.641706 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.641708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:10.642249 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:10.642255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.642633 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.642635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:10.642693 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:10.643168 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:10.643178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:10.644855 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
W20260621 07:57:10.645103 21320 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:10.640191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52617 (local address 127.11.223.190:45057)
0621 07:57:10.640455 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:10.640459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:10.640471 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:57:10.640618 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:10.640620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:10.640661 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:10.640731 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:10.641112 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.641622 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.642341 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.642545 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.642617 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:10.644972 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:52617: BlockingRecv error: recv got EOF from 127.11.223.129:52617 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
Jun 21 07:57:10 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:10 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:10.819010 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:10.813530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:46519)
0621 07:57:10.813817 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:10.813831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:10.813978 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:10.814285 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:10.814288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:10.814500 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:10.814592 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:10.814598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.815356 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.815361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:10.815936 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:10.815944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.816554 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.816557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:10.816641 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:10.817142 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:10.817154 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:10.818828 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
W20260621 07:57:10.819068 21320 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:10.813633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:46519 (local address 127.11.223.190:45057)
0621 07:57:10.813786 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:10.813790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:10.813933 (+ 143us) server_negotiation.cc:407] Connection header received
0621 07:57:10.814119 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:10.814123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:10.814177 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:10.814263 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:10.814710 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.815239 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.816064 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.816442 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.816553 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:10.818956 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:46519: BlockingRecv error: recv got EOF from 127.11.223.130:46519 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 21 07:57:10 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:10 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:11.000967 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:10.995986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:42765)
0621 07:57:10.996424 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:10.996436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:10.996552 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:10.996819 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:10.996822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:10.997034 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:10.997136 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:10.997141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.997807 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.997809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:10.998291 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:10.998297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.998643 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.998646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:10.998698 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:10.999133 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:10.999143 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:11.000832 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":102,"threads_started":1}
W20260621 07:57:11.001072 21320 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:10.996098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:42765 (local address 127.11.223.190:45057)
0621 07:57:10.996240 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:10.996244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:10.996509 (+ 265us) server_negotiation.cc:407] Connection header received
0621 07:57:10.996647 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:10.996651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:10.996703 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:10.996777 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:10.997229 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.997729 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.998377 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:10.998562 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:10.998627 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:11.000962 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:42765: BlockingRecv error: recv got EOF from 127.11.223.131:42765 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 21 07:57:11 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:11 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:11.651369 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:11.645806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:37897)
0621 07:57:11.646056 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:11.646068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:11.646146 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:11.646401 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:11.646403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:11.646561 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:11.646649 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:11.646653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.647339 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.647341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:11.647993 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:11.648001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.648420 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.648423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:11.648525 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:11.649082 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:11.649094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:11.651254 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
W20260621 07:57:11.651520 21324 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:11.645983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:37897 (local address 127.11.223.190:45057)
0621 07:57:11.646196 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:11.646199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:11.646210 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:11.646259 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:11.646261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:11.646297 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:11.646371 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:11.646744 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.647241 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.648088 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.648296 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.648418 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:11.651381 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:37897: BlockingRecv error: recv got EOF from 127.11.223.129:37897 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
Jun 21 07:57:11 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:11 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:11.825151 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:11.819860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:39695)
0621 07:57:11.820135 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:11.820148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:11.820264 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:11.820636 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:11.820639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:11.820898 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:11.821032 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:11.821039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.821911 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.821914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:11.822444 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:11.822451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.822818 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.822820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:11.822877 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:11.823395 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:11.823408 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:11.825045 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
W20260621 07:57:11.825379 21324 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:11.819937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:39695 (local address 127.11.223.190:45057)
0621 07:57:11.820081 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:11.820085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:11.820218 (+ 133us) server_negotiation.cc:407] Connection header received
0621 07:57:11.820433 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:11.820436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:11.820490 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:11.820583 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:11.821153 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.821810 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.822534 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:11.822731 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:11.822799 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:11.825162 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:39695: BlockingRecv error: recv got EOF from 127.11.223.130:39695 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 21 07:57:12 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:12 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:12.007277 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:12.001694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:51473)
0621 07:57:12.002040 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:12.002053 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:12.002142 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:12.002363 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:12.002365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:12.002599 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:12.002721 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:12.002729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.003504 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.003507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:12.004395 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:12.004404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.004863 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.004866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:12.004956 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:12.005481 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:12.005494 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:12.007176 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
W20260621 07:57:12.007411 21324 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:12.001778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:51473 (local address 127.11.223.190:45057)
0621 07:57:12.001914 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:12.001926 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:12.002118 (+ 192us) server_negotiation.cc:407] Connection header received
0621 07:57:12.002204 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:12.002206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:12.002250 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:12.002318 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:12.002845 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.003383 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.004516 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.004757 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.004859 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:12.007300 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:51473: BlockingRecv error: recv got EOF from 127.11.223.131:51473 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 21 07:57:12 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:12 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:12.656932 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:12.652049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:43013)
0621 07:57:12.652330 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:12.652344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:12.652430 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:12.652634 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:12.652636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:12.652822 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:12.652931 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:12.652936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.653574 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.653576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:12.654099 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:12.654105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.654478 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.654481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:12.654537 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:12.655057 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:12.655068 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:12.656821 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":61,"threads_started":1}
W20260621 07:57:12.657084 21328 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:12.652132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:43013 (local address 127.11.223.190:45057)
0621 07:57:12.652435 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:12.652439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:12.652450 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:57:12.652496 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:12.652498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:12.652536 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:12.652600 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:12.653023 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.653494 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.654187 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.654391 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.654460 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:12.656949 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:43013: BlockingRecv error: recv got EOF from 127.11.223.129:43013 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":69,"threads_started":1}
Jun 21 07:57:12 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:12 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:12.830943 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:12.825973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:53241)
0621 07:57:12.826213 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:12.826225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:12.826319 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:12.826553 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:12.826555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:12.826705 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:12.826793 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:12.826797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.827474 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.827476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:12.827999 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:12.828006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.828423 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.828425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:12.828484 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:12.828947 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:12.828961 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:12.830829 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":60,"threads_started":1}
W20260621 07:57:12.831063 21328 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:12.826086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:53241 (local address 127.11.223.190:45057)
0621 07:57:12.826246 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:12.826249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:12.826277 (+ 28us) server_negotiation.cc:407] Connection header received
0621 07:57:12.826417 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:12.826419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:12.826459 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:12.826519 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:12.826897 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.827383 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.828092 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:12.828291 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:12.828380 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:12.830948 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:53241: BlockingRecv error: recv got EOF from 127.11.223.130:53241 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 21 07:57:13 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:13 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:13.013406 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:13.008004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:38853)
0621 07:57:13.008288 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:13.008302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:13.008446 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:13.008682 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:13.008685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:13.008880 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:13.008974 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:13.008980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.009820 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.009823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:13.010587 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:13.010601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.011088 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.011090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:13.011149 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:13.011616 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:13.011627 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:13.013301 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
W20260621 07:57:13.013541 21328 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:13.008069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:38853 (local address 127.11.223.190:45057)
0621 07:57:13.008224 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:13.008229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:13.008395 (+ 166us) server_negotiation.cc:407] Connection header received
0621 07:57:13.008526 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:13.008528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:13.008576 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:13.008666 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:13.009083 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.009689 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.010740 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.010991 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.011070 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:13.013427 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:38853: BlockingRecv error: recv got EOF from 127.11.223.131:38853 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 21 07:57:13 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:13 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:13.662922 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:13.657792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:51137)
0621 07:57:13.658094 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:13.658105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:13.658202 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:13.658422 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:13.658424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:13.658605 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:13.658690 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:13.658694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.659333 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.659335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:13.659889 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:13.659895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.660280 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.660297 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:13.660396 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:13.660921 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:13.660932 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:13.662816 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":83,"threads_started":1}
W20260621 07:57:13.663084 21332 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:13.657815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:51137 (local address 127.11.223.190:45057)
0621 07:57:13.658034 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:13.658037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:13.658164 (+ 127us) server_negotiation.cc:407] Connection header received
0621 07:57:13.658279 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:13.658281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:13.658322 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:13.658388 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:13.658773 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.659243 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.659978 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.660182 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.660251 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:13.662948 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:51137: BlockingRecv error: recv got EOF from 127.11.223.129:51137 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
Jun 21 07:57:13 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:13 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:13.836993 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:13.831755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:42939)
0621 07:57:13.832079 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:13.832096 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:13.832221 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:13.832513 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:13.832516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:13.832717 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:13.832799 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:13.832804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.833518 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.833522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:13.834086 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:13.834094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.834565 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.834569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:13.834657 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:13.835132 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:13.835143 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:13.836850 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":114,"threads_started":1}
W20260621 07:57:13.837088 21332 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:13.831855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:42939 (local address 127.11.223.190:45057)
0621 07:57:13.832026 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:13.832029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:13.832176 (+ 147us) server_negotiation.cc:407] Connection header received
0621 07:57:13.832349 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:13.832352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:13.832405 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:13.832485 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:13.832919 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.833410 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.834205 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:13.834438 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:13.834563 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:13.836984 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:42939: BlockingRecv error: recv got EOF from 127.11.223.130:42939 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 21 07:57:14 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:14 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:14.019470 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:14.014252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:51925)
0621 07:57:14.014533 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:14.014546 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:14.014655 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:14.014900 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:14.014902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:14.015077 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:14.015180 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:14.015184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.015911 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.015913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:14.016460 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:14.016468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.016873 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.016875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:14.016940 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:14.017463 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:14.017475 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:14.019290 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":106,"threads_started":1}
W20260621 07:57:14.019518 21332 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:14.014322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:51925 (local address 127.11.223.190:45057)
0621 07:57:14.014471 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:14.014477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:14.014635 (+ 158us) server_negotiation.cc:407] Connection header received
0621 07:57:14.014731 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:14.014733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:14.014775 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:14.014858 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:14.015279 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.015818 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.016552 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.016765 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.016841 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:14.019416 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:51925: BlockingRecv error: recv got EOF from 127.11.223.131:51925 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 21 07:57:14 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:14 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:14.669075 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:14.663624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:52901)
0621 07:57:14.663897 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:14.663910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:14.664005 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:14.664233 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:14.664235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:14.664429 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:14.664518 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:14.664523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.665391 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.665395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:14.666134 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:14.666143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.666541 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.666543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:14.666598 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:14.667141 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:14.667152 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:14.668913 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":65,"threads_started":1}
W20260621 07:57:14.669135 21336 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:14.663677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52901 (local address 127.11.223.190:45057)
0621 07:57:14.663918 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:14.663921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:14.663976 (+ 55us) server_negotiation.cc:407] Connection header received
0621 07:57:14.664083 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:14.664085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:14.664127 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:14.664200 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:14.664625 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.665281 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.666233 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.666451 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.666528 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:14.669022 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:52901: BlockingRecv error: recv got EOF from 127.11.223.129:52901 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
Jun 21 07:57:14 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:14 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:14.842731 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:14.837744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:48359)
0621 07:57:14.838006 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:14.838018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:14.838113 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:14.838374 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:14.838377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:14.838569 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:14.838662 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:14.838667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.839363 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.839365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:14.839909 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:14.839915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.840309 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.840367 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:14.840441 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:14.840927 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:14.840938 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:14.842623 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
W20260621 07:57:14.842871 21336 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:14.837800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:48359 (local address 127.11.223.190:45057)
0621 07:57:14.837936 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:14.837939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:14.838082 (+ 143us) server_negotiation.cc:407] Connection header received
0621 07:57:14.838187 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:14.838189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:14.838232 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:14.838316 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:14.838760 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.839272 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.840006 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:14.840216 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:14.840293 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:14.842751 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:48359: BlockingRecv error: recv got EOF from 127.11.223.130:48359 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 21 07:57:15 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:15 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:15.029222 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:15.020374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:47677)
0621 07:57:15.020643 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:15.020656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:15.020748 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:15.020994 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:15.020996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:15.021172 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:15.021263 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:15.021268 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.021948 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.021950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:15.022476 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:15.022481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.022860 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.022862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:15.022919 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:15.027073 (+ 4154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:15.027084 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:15.029067 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":103,"threads_started":1}
W20260621 07:57:15.029270 21336 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:15.020422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:47677 (local address 127.11.223.190:45057)
0621 07:57:15.020550 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:15.020553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:15.020720 (+ 167us) server_negotiation.cc:407] Connection header received
0621 07:57:15.020823 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:15.020825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:15.020890 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:15.020956 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:15.021356 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.021860 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.022566 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.022764 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.022833 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:15.029168 (+ 6335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:47677: BlockingRecv error: recv got EOF from 127.11.223.131:47677 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 21 07:57:15 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:15 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:15.674814 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:15.669818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:35529)
0621 07:57:15.670125 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:15.670139 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:15.670226 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:15.670456 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:15.670458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:15.670633 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:15.670719 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:15.670724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.671429 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.671431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:15.671967 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:15.671973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.672434 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.672436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:15.672492 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:15.673018 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:15.673029 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:15.674709 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":65,"threads_started":1}
W20260621 07:57:15.674988 21340 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:15.669916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:35529 (local address 127.11.223.190:45057)
0621 07:57:15.670163 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:15.670166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:15.670183 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:57:15.670307 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:15.670309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:15.670350 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:15.670421 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:15.670809 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.671343 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.672061 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.672274 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.672426 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:15.674826 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:35529: BlockingRecv error: recv got EOF from 127.11.223.129:35529 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
Jun 21 07:57:15 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:15 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:15.849087 21341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:15.843646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:43191)
0621 07:57:15.844007 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:15.844019 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:15.844155 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:15.844444 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:15.844447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:15.844653 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:15.844737 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:15.844742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.845491 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.845497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:15.846045 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:15.846052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.846504 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.846507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:15.846587 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:15.847057 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:15.847069 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:15.848983 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":82,"threads_started":1}
W20260621 07:57:15.849282 21340 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:15.843765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:43191 (local address 127.11.223.190:45057)
0621 07:57:15.843947 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:15.843953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:15.844097 (+ 144us) server_negotiation.cc:407] Connection header received
0621 07:57:15.844254 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:15.844258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:15.844330 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:15.844429 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:15.844850 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.845382 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.846158 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:15.846378 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:15.846505 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:15.849110 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:43191: BlockingRecv error: recv got EOF from 127.11.223.130:43191 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Jun 21 07:57:16 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:16 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:16.035205 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:16.030024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:43703)
0621 07:57:16.030334 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:16.030349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:16.030453 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:16.030635 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:16.030636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:16.030803 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:16.030899 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:16.030904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.031715 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.031717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:16.032263 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:16.032269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.032799 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.032802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:16.032870 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:16.033306 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:16.033316 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:16.035041 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":118,"threads_started":1}
W20260621 07:57:16.035247 21340 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:16.030065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:43703 (local address 127.11.223.190:45057)
0621 07:57:16.030222 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:16.030226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:16.030428 (+ 202us) server_negotiation.cc:407] Connection header received
0621 07:57:16.030516 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:16.030518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:16.030564 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:16.030625 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:16.031013 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.031623 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.032432 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.032697 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.032789 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:16.035150 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:43703: BlockingRecv error: recv got EOF from 127.11.223.131:43703 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 21 07:57:16 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:16 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:16.680756 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:16.675468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:42791)
0621 07:57:16.675728 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:16.675740 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:16.675818 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:16.676033 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:16.676035 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:16.676194 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:16.676295 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:16.676299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.677137 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.677142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:16.677715 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:16.677723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.678192 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.678195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:16.678302 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:16.678766 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:16.678778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:16.680641 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
W20260621 07:57:16.680908 21344 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:16.675609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:42791 (local address 127.11.223.190:45057)
0621 07:57:16.675799 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:16.675802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:16.675815 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:57:16.675894 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:16.675896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:16.675933 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:16.676003 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:16.676443 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.677005 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.677836 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.678071 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.678187 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:16.680778 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:42791: BlockingRecv error: recv got EOF from 127.11.223.129:42791 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":58,"threads_started":1}
Jun 21 07:57:16 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:16 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:16.855597 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:16.849998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:41081)
0621 07:57:16.850246 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:16.850259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:16.850356 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:16.850586 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:16.850588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:16.850783 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:16.850872 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:16.850877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.851737 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.851739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:16.852300 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:16.852307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.852748 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.852751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:16.852810 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:16.853416 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:16.853434 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:16.855434 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
W20260621 07:57:16.855651 21344 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:16.850067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:41081 (local address 127.11.223.190:45057)
0621 07:57:16.850217 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:16.850221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:16.850328 (+ 107us) server_negotiation.cc:407] Connection header received
0621 07:57:16.850445 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:16.850447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:16.850490 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:16.850554 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:16.850961 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.851640 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.852444 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:16.852650 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:16.852732 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:16.855547 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:41081: BlockingRecv error: recv got EOF from 127.11.223.130:41081 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 21 07:57:17 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:17 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:17.040736 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:17.035969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:51919)
0621 07:57:17.036227 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:17.036240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:17.036371 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:17.036624 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:17.036625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:17.036791 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:17.036876 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:17.036881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.037513 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.037515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:17.038000 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:17.038006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.038354 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.038356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:17.038419 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:17.038890 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:17.038900 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:17.040636 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
W20260621 07:57:17.040889 21344 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:17.036012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:51919 (local address 127.11.223.190:45057)
0621 07:57:17.036147 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:17.036150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:17.036303 (+ 153us) server_negotiation.cc:407] Connection header received
0621 07:57:17.036493 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:17.036495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:17.036549 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:17.036612 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:17.036960 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.037428 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.038088 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.038271 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.038335 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:17.040748 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:51919: BlockingRecv error: recv got EOF from 127.11.223.131:51919 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 21 07:57:17 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:17 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:17.686646 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:17.681492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:52899)
0621 07:57:17.681752 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:17.681765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:17.681861 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:17.682083 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:17.682084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:17.682260 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:17.682351 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:17.682368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.683138 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.683141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:17.683774 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:17.683784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.684208 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.684210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:17.684269 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:17.684748 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:17.684760 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:17.686542 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
W20260621 07:57:17.686800 21348 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:17.681551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52899 (local address 127.11.223.190:45057)
0621 07:57:17.681777 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:17.681781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:17.681834 (+ 53us) server_negotiation.cc:407] Connection header received
0621 07:57:17.681937 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:17.681939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:17.681981 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:17.682055 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:17.682453 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.683045 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.683892 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.684116 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.684192 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:17.686661 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:52899: BlockingRecv error: recv got EOF from 127.11.223.129:52899 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":57,"threads_started":1}
Jun 21 07:57:17 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:17 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:17.862396 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:17.856628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:55873)
0621 07:57:17.856899 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:17.856911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:17.857043 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:17.857305 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:17.857308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:17.857574 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:17.857664 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:17.857671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.858446 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.858451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:17.858998 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:17.859006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.859445 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.859448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:17.859524 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:17.860115 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:17.860128 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:17.862208 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
W20260621 07:57:17.862460 21348 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:17.856688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:55873 (local address 127.11.223.190:45057)
0621 07:57:17.856852 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:17.856857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:17.856984 (+ 127us) server_negotiation.cc:407] Connection header received
0621 07:57:17.857138 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:17.857141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:17.857197 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:17.857286 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:17.857773 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.858323 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.859112 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:17.859335 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:17.859459 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:17.862350 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:55873: BlockingRecv error: recv got EOF from 127.11.223.130:55873 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 21 07:57:18 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:18 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:18.046578 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:18.041487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:54347)
0621 07:57:18.041750 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:18.041763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:18.041886 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:18.042154 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:18.042157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:18.042355 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:18.042441 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:18.042446 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.043210 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.043212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:18.043732 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:18.043738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.044112 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.044114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:18.044170 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:18.044725 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:18.044736 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:18.046458 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
W20260621 07:57:18.046692 21348 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:18.041541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:54347 (local address 127.11.223.190:45057)
0621 07:57:18.041679 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:18.041685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:18.041834 (+ 149us) server_negotiation.cc:407] Connection header received
0621 07:57:18.041978 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:18.041982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:18.042038 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:18.042135 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:18.042572 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.043105 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.043828 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.044026 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.044095 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:18.046581 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:54347: BlockingRecv error: recv got EOF from 127.11.223.131:54347 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 21 07:57:18 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:18 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:18.692826 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:18.687507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:44329)
0621 07:57:18.687817 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:18.687830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:18.687909 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:18.688181 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:18.688183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:18.688393 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:18.688493 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:18.688498 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.689237 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.689239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:18.689785 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:18.689792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.690172 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.690174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:18.690232 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:18.690797 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:18.690811 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:18.692652 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":76,"threads_started":1}
W20260621 07:57:18.692898 21352 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:18.687576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:44329 (local address 127.11.223.190:45057)
0621 07:57:18.687835 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:18.687843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:18.687877 (+ 34us) server_negotiation.cc:407] Connection header received
0621 07:57:18.688001 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:18.688003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:18.688042 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:18.688126 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:18.688583 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.689115 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.689879 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.690084 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.690155 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:18.692759 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:44329: BlockingRecv error: recv got EOF from 127.11.223.129:44329 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
Jun 21 07:57:18 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:18 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:18.868228 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:18.863214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:44657)
0621 07:57:18.863465 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:18.863477 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:18.863561 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:18.863776 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:18.863778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:18.863951 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:18.864053 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:18.864058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.864723 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.864725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:18.865251 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:18.865258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.865655 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.865657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:18.865723 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:18.866197 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:18.866207 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:18.868116 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
W20260621 07:57:18.868368 21352 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:18.863267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:44657 (local address 127.11.223.190:45057)
0621 07:57:18.863400 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:18.863403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:18.863554 (+ 151us) server_negotiation.cc:407] Connection header received
0621 07:57:18.863619 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:18.863621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:18.863677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:18.863736 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:18.864146 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.864640 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.865364 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:18.865569 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:18.865638 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:18.868240 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:44657: BlockingRecv error: recv got EOF from 127.11.223.130:44657 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 21 07:57:19 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:19 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:19.053210 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:19.047580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:52859)
0621 07:57:19.047873 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:19.047888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:19.048006 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:19.048460 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:19.048463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:19.048713 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:19.048809 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:19.048816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.049643 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.049649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:19.050225 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:19.050232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.050690 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.050694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:19.050783 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:19.051248 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:19.051259 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:19.053045 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":101,"threads_started":1}
W20260621 07:57:19.053278 21352 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:19.047715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:52859 (local address 127.11.223.190:45057)
0621 07:57:19.047868 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:19.047872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:19.047959 (+ 87us) server_negotiation.cc:407] Connection header received
0621 07:57:19.048188 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:19.048190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:19.048296 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:19.048406 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:19.048940 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.049540 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.050344 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.050578 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.050689 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:19.053157 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:52859: BlockingRecv error: recv got EOF from 127.11.223.131:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 21 07:57:19 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:19 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:19.698840 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:19.693570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:59289)
0621 07:57:19.693826 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:19.693838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:19.693920 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:19.694125 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:19.694127 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:19.694291 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:19.694379 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:19.694384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.695046 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.695050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:19.695820 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:19.695830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.696226 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.696228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:19.696285 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:19.696840 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:19.696853 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:19.698721 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":65,"threads_started":1}
W20260621 07:57:19.698987 21356 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:19.693643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:59289 (local address 127.11.223.190:45057)
0621 07:57:19.693887 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:19.693890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:19.693902 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:57:19.693992 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:19.693994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:19.694033 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:19.694095 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:19.694478 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.694962 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.695921 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.696131 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.696204 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:19.698866 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:59289: BlockingRecv error: recv got EOF from 127.11.223.129:59289 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
Jun 21 07:57:19 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:19 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:19.874245 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:19.869026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:44909)
0621 07:57:19.869327 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:19.869341 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:19.869437 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:19.869655 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:19.869657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:19.869829 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:19.869917 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:19.869922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.870655 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.870657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:19.871209 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:19.871216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.871630 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.871633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:19.871702 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:19.872164 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:19.872181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:19.874129 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":64,"threads_started":1}
W20260621 07:57:19.874374 21356 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:19.869110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:44909 (local address 127.11.223.190:45057)
0621 07:57:19.869325 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:19.869328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:19.869408 (+ 80us) server_negotiation.cc:407] Connection header received
0621 07:57:19.869513 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:19.869515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:19.869556 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:19.869620 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:19.870009 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.870544 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.871303 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:19.871509 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:19.871582 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:19.874259 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:44909: BlockingRecv error: recv got EOF from 127.11.223.130:44909 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Jun 21 07:57:20 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:20 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:20.063483 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:20.053939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:58895)
0621 07:57:20.054189 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:20.054207 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:20.054306 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:20.054523 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:20.054525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:20.054704 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:20.054796 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:20.054801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.055472 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.055474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:20.056083 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:20.056093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.056686 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.056693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:20.056781 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:20.061546 (+ 4765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:20.061558 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:20.063327 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
W20260621 07:57:20.063531 21356 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:20.053972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:58895 (local address 127.11.223.190:45057)
0621 07:57:20.054108 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:20.054111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:20.054274 (+ 163us) server_negotiation.cc:407] Connection header received
0621 07:57:20.054382 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:20.054384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:20.054428 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:20.054492 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:20.054887 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.055385 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.056206 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.056539 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.056652 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:20.063433 (+ 6781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:58895: BlockingRecv error: recv got EOF from 127.11.223.131:58895 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 21 07:57:20 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:20 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:20.705025 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:20.699616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:33977)
0621 07:57:20.699894 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:20.699906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:20.699989 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:20.700292 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:20.700294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:20.700509 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:20.700599 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:20.700604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.701494 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.701500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:20.702051 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:20.702059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.702524 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.702527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:20.702610 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:20.703073 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:20.703084 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:20.704863 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
W20260621 07:57:20.705097 21360 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:20.699671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:33977 (local address 127.11.223.190:45057)
0621 07:57:20.699926 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:20.699928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:20.699942 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:57:20.700085 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:20.700087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:20.700128 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:20.700234 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:20.700705 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.701379 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.702170 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.702401 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.702520 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:20.704976 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:33977: BlockingRecv error: recv got EOF from 127.11.223.129:33977 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
Jun 21 07:57:20 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:20 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:20.880276 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:20.875284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:47565)
0621 07:57:20.875556 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:20.875569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:20.875675 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:20.875915 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:20.875917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:20.876084 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:20.876174 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:20.876179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.876835 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.876837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:20.877371 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:20.877379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.877736 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.877739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:20.877793 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:20.878300 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:20.878311 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:20.880092 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":110,"threads_started":1}
W20260621 07:57:20.880307 21360 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:20.875326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:47565 (local address 127.11.223.190:45057)
0621 07:57:20.875468 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:20.875472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:20.875648 (+ 176us) server_negotiation.cc:407] Connection header received
0621 07:57:20.875765 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:20.875767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:20.875809 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:20.875868 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:20.876271 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.876756 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.877462 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:20.877646 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:20.877711 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:20.880191 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:47565: BlockingRecv error: recv got EOF from 127.11.223.130:47565 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 21 07:57:21 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:21 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:21.069257 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:21.064190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:42839)
0621 07:57:21.064597 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:21.064610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:21.064736 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:21.065002 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:21.065005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:21.065200 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:21.065290 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:21.065295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.065981 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.065986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:21.066498 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:21.066505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.066920 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.066923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:21.067000 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:21.067437 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:21.067447 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:21.069127 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":128,"threads_started":1}
W20260621 07:57:21.069377 21360 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:21.064283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:42839 (local address 127.11.223.190:45057)
0621 07:57:21.064440 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:21.064445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:21.064686 (+ 241us) server_negotiation.cc:407] Connection header received
0621 07:57:21.064851 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:21.064855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:21.064905 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:21.064982 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:21.065397 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.065881 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.066609 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.066816 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.066916 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:21.069265 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:42839: BlockingRecv error: recv got EOF from 127.11.223.131:42839 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 21 07:57:21 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:21 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:21.710747 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:21.705782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:36191)
0621 07:57:21.706035 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:21.706047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:21.706128 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:21.706365 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:21.706367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:21.706540 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:21.706644 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:21.706649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.707351 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.707353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:21.707882 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:21.707888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.708266 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.708268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:21.708411 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:21.708868 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:21.708879 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:21.710630 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
W20260621 07:57:21.710917 21364 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:21.705858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:36191 (local address 127.11.223.190:45057)
0621 07:57:21.706076 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:21.706079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:21.706091 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:57:21.706215 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:21.706217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:21.706259 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:21.706330 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:21.706748 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.707267 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.707975 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.708178 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.708248 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:21.710756 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:36191: BlockingRecv error: recv got EOF from 127.11.223.129:36191 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
Jun 21 07:57:21 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:21 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:21.886637 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:21.881112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:36631)
0621 07:57:21.881417 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:21.881429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:21.881523 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:21.881766 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:21.881768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:21.881943 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:21.882031 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:21.882036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.882861 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.882866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:21.883758 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:21.883774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.884184 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.884186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:21.884244 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:21.884711 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:21.884722 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:21.886470 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":122,"threads_started":1}
W20260621 07:57:21.886672 21364 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:21.881268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:36631 (local address 127.11.223.190:45057)
0621 07:57:21.881417 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:21.881420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:21.881494 (+ 74us) server_negotiation.cc:407] Connection header received
0621 07:57:21.881611 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:21.881613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:21.881656 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:21.881726 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:21.882121 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.882770 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.883881 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:21.884095 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:21.884172 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:21.886573 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:36631: BlockingRecv error: recv got EOF from 127.11.223.130:36631 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 21 07:57:22 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:22 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:22.074913 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:22.070039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:35789)
0621 07:57:22.070304 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:22.070317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:22.070417 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:22.070615 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:22.070617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:22.070790 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:22.070875 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:22.070879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.071550 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.071552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:22.072107 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:22.072113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.072543 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.072545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:22.072598 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:22.073031 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:22.073042 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:22.074752 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
W20260621 07:57:22.074960 21364 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:22.070113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:35789 (local address 127.11.223.190:45057)
0621 07:57:22.070247 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:22.070251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:22.070389 (+ 138us) server_negotiation.cc:407] Connection header received
0621 07:57:22.070476 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:22.070478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:22.070518 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:22.070580 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:22.070960 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.071464 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.072206 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.072431 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.072528 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:22.074855 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:35789: BlockingRecv error: recv got EOF from 127.11.223.131:35789 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 21 07:57:22 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:22 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:22.716488 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:22.711450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:41363)
0621 07:57:22.711711 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:22.711724 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:22.711810 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:22.712021 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:22.712023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:22.712192 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:22.712281 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:22.712286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.712997 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.712999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:22.713518 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:22.713524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.713903 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.713905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:22.713962 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:22.714484 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:22.714499 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:22.716369 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
W20260621 07:57:22.716602 21368 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:22.711499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:41363 (local address 127.11.223.190:45057)
0621 07:57:22.711791 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:22.711794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:22.711807 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:57:22.711887 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:22.711889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:22.711928 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:22.711994 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:22.712395 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.712914 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.713609 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.713819 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.713887 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:22.716478 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:41363: BlockingRecv error: recv got EOF from 127.11.223.129:41363 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":58,"threads_started":1}
Jun 21 07:57:22 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:22 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:22.893020 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:22.887514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:33723)
0621 07:57:22.887850 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:22.887879 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:22.888003 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:22.888271 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:22.888274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:22.888496 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:22.888584 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:22.888589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.889470 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.889473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:22.890192 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:22.890202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.890590 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.890593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:22.890651 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:22.891094 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:22.891105 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:22.892888 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
W20260621 07:57:22.893105 21368 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:22.887581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:33723 (local address 127.11.223.190:45057)
0621 07:57:22.887823 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:22.887828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:22.887985 (+ 157us) server_negotiation.cc:407] Connection header received
0621 07:57:22.888085 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:22.888089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:22.888148 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:22.888246 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:22.888702 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.889363 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.890294 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:22.890499 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:22.890576 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:22.893015 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:33723: BlockingRecv error: recv got EOF from 127.11.223.130:33723 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Jun 21 07:57:23 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:23 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.131@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:23.080927 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:23.075991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.131:46723)
0621 07:57:23.076283 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:23.076296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:23.076410 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:23.076646 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:23.076648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:23.076818 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:23.076908 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:23.076913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.077596 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.077597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:23.078094 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:23.078099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.078473 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.078476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:23.078534 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:23.079026 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:23.079036 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:23.080812 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
W20260621 07:57:23.081056 21368 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:23.076078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:46723 (local address 127.11.223.190:45057)
0621 07:57:23.076217 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:23.076220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:23.076371 (+ 151us) server_negotiation.cc:407] Connection header received
0621 07:57:23.076497 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:23.076499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:23.076543 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:23.076611 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:23.076999 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.077488 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.078176 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.078372 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.078440 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:23.080936 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.131:46723: BlockingRecv error: recv got EOF from 127.11.223.131:46723 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 21 07:57:23 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:23 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.129@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:23.722689 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:23.717394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.129:49219)
0621 07:57:23.717690 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:23.717708 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:23.717825 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:23.718073 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:23.718076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:23.718289 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:23.718375 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:23.718380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.719149 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.719155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:23.719721 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:23.719729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.720182 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.720185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:23.720261 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:23.720770 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:23.720782 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:23.722501 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
W20260621 07:57:23.722750 21372 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:23.717464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:49219 (local address 127.11.223.190:45057)
0621 07:57:23.717726 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:23.717731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:23.717772 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:57:23.717908 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:23.717911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:23.717953 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:23.718055 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:23.718499 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.719038 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.719843 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.720069 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.720187 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:23.722621 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.129:49219: BlockingRecv error: recv got EOF from 127.11.223.129:49219 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":57,"threads_started":1}
Jun 21 07:57:23 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
Jun 21 07:57:23 dist-test-slave-0pn3 krb5kdc[20314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.223.130@KRBTEST.COM for oryx/127.11.223.190@KRBTEST.COM, Server not found in Kerberos database
I20260621 07:57:23.898782 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:23.893880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:45057 (local address 127.11.223.130:46133)
0621 07:57:23.894184 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:23.894197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:23.894288 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:23.894514 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:23.894516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:23.894697 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:23.894789 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:23.894794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.895535 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.895537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:23.896095 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:23.896102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.896495 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.896498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:23.896557 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:23.897008 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:23.897018 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:23.898664 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.223.190:45057: Server oryx/127.11.223.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":113,"threads_started":1}
W20260621 07:57:23.898907 21372 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:23.893949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:46133 (local address 127.11.223.190:45057)
0621 07:57:23.894126 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:23.894132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:23.894268 (+ 136us) server_negotiation.cc:407] Connection header received
0621 07:57:23.894367 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:23.894369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:23.894413 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:23.894479 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:23.894884 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.895429 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.896191 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:23.896409 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:23.896480 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:23.898795 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.223.130:46133: BlockingRecv error: recv got EOF from 127.11.223.130:46133 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 07:57:23.968189 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20897
I20260621 07:57:23.973799 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21049
I20260621 07:57:23.978813 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21181
I20260621 07:57:23.983584 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 20827
2026-06-21T07:57:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17827 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-21T07:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:24Z Disabled control of system clock
I20260621 07:57:24.012893 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:42541
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:42541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260621 07:57:24.121219 21377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:24.121450 21377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:24.121536 21377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:24.125586 21377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:24.125636 21377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:24.125651 21377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:24.125671 21377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:24.125686 21377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:24.130431 21377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:42541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:42541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:24.131414 21377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:24.132781 21377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:24.140866 21382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.140846 21385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.140846 21383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:24.140892 21377 server_base.cc:1034] running on GCE node
I20260621 07:57:24.141680 21377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:24.142694 21377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:24.143889 21377 hybrid_clock.cc:648] HybridClock initialized: now 1782028644143868 us; error 43 us; skew 500 ppm
I20260621 07:57:24.145735 21377 webserver.cc:460] Webserver started at http://127.11.223.190:35767/ using document root <none> and password file <none>
I20260621 07:57:24.146279 21377 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:24.146366 21377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:24.146613 21377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:24.148613 21377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "afbd70b59cd746828f9dd51c321eb709"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "94a48e51672a2601a14ad488bc09e3e7"
server_key_iv: "2847cedf0b4dafa377d9e40eef5005aa"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.149180 21377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "afbd70b59cd746828f9dd51c321eb709"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "94a48e51672a2601a14ad488bc09e3e7"
server_key_iv: "2847cedf0b4dafa377d9e40eef5005aa"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.152745 21377 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260621 07:57:24.154971 21391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.156139 21377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:24.156289 21377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "afbd70b59cd746828f9dd51c321eb709"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "94a48e51672a2601a14ad488bc09e3e7"
server_key_iv: "2847cedf0b4dafa377d9e40eef5005aa"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.156453 21377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:24.170615 21377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:24.171227 21377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:24.171414 21377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:24.177862 21377 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:42541
I20260621 07:57:24.177872 21443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:42541 every 8 connection(s)
I20260621 07:57:24.178718 21377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:57:24.181806 21444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:24.186877 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709: Bootstrap starting.
I20260621 07:57:24.187831 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21377
I20260621 07:57:24.187951 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:57:24.188273 12158 external_mini_cluster.cc:1428] Setting key be8ea47b4d000c2b8b60fea29623c9cd
I20260621 07:57:24.189724 21444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:24.190428 21444 log.cc:824] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:24.192202 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709: No bootstrap required, opened a new log
I20260621 07:57:24.195799 21444 raft_consensus.cc:348] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afbd70b59cd746828f9dd51c321eb709" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42541 } }
I20260621 07:57:24.196039 21444 raft_consensus.cc:374] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:24.196115 21444 raft_consensus.cc:729] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afbd70b59cd746828f9dd51c321eb709, State: Initialized, Role: FOLLOWER
I20260621 07:57:24.196609 21444 consensus_queue.cc:260] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afbd70b59cd746828f9dd51c321eb709" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42541 } }
I20260621 07:57:24.196771 21444 raft_consensus.cc:388] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:57:24.196861 21444 raft_consensus.cc:482] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:57:24.196959 21444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:24.197104 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.189375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.11.223.190:42541)
0621 07:57:24.189994 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:24.190008 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:24.190041 (+ 33us) server_negotiation.cc:407] Connection header received
0621 07:57:24.190672 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:24.190693 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:24.191096 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:24.191464 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:24.191861 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.192897 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.193766 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.194238 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.194413 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.195277 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:24.195282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:24.195294 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:24.195646 (+ 352us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:24.195699 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:24.195718 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:24.195826 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:24.195940 (+ 114us) server_negotiation.cc:299] Negotiation successful
0621 07:57:24.196240 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260621 07:57:24.198035 21444 raft_consensus.cc:504] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afbd70b59cd746828f9dd51c321eb709" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42541 } }
I20260621 07:57:24.198380 21444 leader_election.cc:302] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afbd70b59cd746828f9dd51c321eb709; no voters:
I20260621 07:57:24.198704 21444 leader_election.cc:288] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:57:24.198766 21449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:24.198972 21449 raft_consensus.cc:686] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [term 1 LEADER]: Becoming Leader. State: Replica: afbd70b59cd746828f9dd51c321eb709, State: Running, Role: LEADER
I20260621 07:57:24.199576 21449 consensus_queue.cc:237] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afbd70b59cd746828f9dd51c321eb709" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42541 } }
I20260621 07:57:24.199915 21444 sys_catalog.cc:564] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:57:24.200732 21449 sys_catalog.cc:455] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afbd70b59cd746828f9dd51c321eb709. Latest consensus state: current_term: 1 leader_uuid: "afbd70b59cd746828f9dd51c321eb709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afbd70b59cd746828f9dd51c321eb709" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42541 } } }
I20260621 07:57:24.200865 21449 sys_catalog.cc:458] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:24.201205 21450 sys_catalog.cc:455] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afbd70b59cd746828f9dd51c321eb709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afbd70b59cd746828f9dd51c321eb709" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 42541 } } }
I20260621 07:57:24.201313 21450 sys_catalog.cc:458] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709 [sys.catalog]: This master's current role is: LEADER
W20260621 07:57:24.204241 21464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:57:24.204331 21464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:57:24.204463 21458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:24.205098 21458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:24.210196 21458 catalog_manager.cc:1347] Generated new cluster ID: 0ea4e1536f974c7489cd215cd19e620d
I20260621 07:57:24.210266 21458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:24.235219 21458 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:57:24.236394 21458 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:24.258096 21458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afbd70b59cd746828f9dd51c321eb709: Generated new TSK 0
I20260621 07:57:24.258749 21458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:57:24.262470 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42541
--builtin_ntp_servers=127.11.223.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260621 07:57:24.367920 21468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:24.368129 21468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:24.368161 21468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:24.372066 21468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:24.372114 21468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:24.372210 21468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:57:24.377038 21468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:24.377966 21468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:24.379139 21468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:24.387928 21473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.387941 21476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.387944 21474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:24.388195 21468 server_base.cc:1034] running on GCE node
I20260621 07:57:24.388630 21468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:24.389235 21468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:24.390388 21468 hybrid_clock.cc:648] HybridClock initialized: now 1782028644390371 us; error 30 us; skew 500 ppm
I20260621 07:57:24.391996 21468 webserver.cc:460] Webserver started at http://127.11.223.129:45053/ using document root <none> and password file <none>
I20260621 07:57:24.392577 21468 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:24.392644 21468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:24.392876 21468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:24.394749 21468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "8272531e6d19472eb78736fb41bda196"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "ffc6fc482cdd8c9286cfaffa81533b04"
server_key_iv: "af13f058d851899f850c0c764158e339"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.395252 21468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "8272531e6d19472eb78736fb41bda196"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "ffc6fc482cdd8c9286cfaffa81533b04"
server_key_iv: "af13f058d851899f850c0c764158e339"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.398710 21468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260621 07:57:24.401072 21482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.402268 21468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:57:24.402374 21468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "8272531e6d19472eb78736fb41bda196"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "ffc6fc482cdd8c9286cfaffa81533b04"
server_key_iv: "af13f058d851899f850c0c764158e339"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.402469 21468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:24.419299 21468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:24.419920 21468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:24.420101 21468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:24.420665 21468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:24.421523 21468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:24.421574 21468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.421648 21468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:24.421685 21468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.430008 21468 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:42481
I20260621 07:57:24.430040 21595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:42481 every 8 connection(s)
I20260621 07:57:24.430850 21468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:57:24.438525 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21468
I20260621 07:57:24.438680 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:57:24.439076 12158 external_mini_cluster.cc:1428] Setting key d5ecd66206f7a6b8ace585d0ab79112e
I20260621 07:57:24.439800 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.432587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:49851 (local address 127.11.223.190:42541)
0621 07:57:24.432734 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:24.432740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:24.433791 (+ 1051us) server_negotiation.cc:407] Connection header received
0621 07:57:24.435112 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:24.435115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:24.435163 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:24.435246 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:24.436109 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.436711 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.437797 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.438011 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.438082 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.439310 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:24.439313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:24.439315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:24.439327 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:24.439343 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:24.439351 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:24.439408 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:24.439595 (+ 187us) server_negotiation.cc:299] Negotiation successful
0621 07:57:24.439671 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 07:57:24.440814 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.433025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:42541 (local address 127.11.223.129:49851)
0621 07:57:24.433656 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:24.433695 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:24.434931 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:24.435399 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:24.435405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:24.435487 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:24.435995 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:24.436006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.436842 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.436844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:24.437698 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:24.437707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.438128 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.438130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:24.438330 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.439029 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:57:24.439032 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:24.439083 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:57:24.439163 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:24.439433 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:24.439442 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:24.439447 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 07:57:24.439571 (+ 124us) client_negotiation.cc:240] Negotiation successful
0621 07:57:24.439842 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":104,"threads_started":1}
I20260621 07:57:24.441210 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42541
--builtin_ntp_servers=127.11.223.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260621 07:57:24.441963 21596 heartbeater.cc:347] Connected to a master server at 127.11.223.190:42541
I20260621 07:57:24.442195 21596 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:24.442744 21596 heartbeater.cc:511] Master 127.11.223.190:42541 requested a full tablet report, sending...
I20260621 07:57:24.444244 21408 ts_manager.cc:194] Registered new tserver with Master: 8272531e6d19472eb78736fb41bda196 (127.11.223.129:42481)
I20260621 07:57:24.445366 21408 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.129:49851
W20260621 07:57:24.547823 21599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:24.548081 21599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:24.548154 21599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:24.552146 21599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:24.552223 21599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:24.552322 21599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:57:24.556944 21599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:24.557898 21599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:24.559134 21599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:24.567730 21607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.567739 21605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.567790 21604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:24.567863 21599 server_base.cc:1034] running on GCE node
I20260621 07:57:24.568473 21599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:24.569039 21599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:24.570222 21599 hybrid_clock.cc:648] HybridClock initialized: now 1782028644570187 us; error 42 us; skew 500 ppm
I20260621 07:57:24.571797 21599 webserver.cc:460] Webserver started at http://127.11.223.130:43379/ using document root <none> and password file <none>
I20260621 07:57:24.572376 21599 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:24.572464 21599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:24.572700 21599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:24.574465 21599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "b28f3b2633ce441988bf131bd7e7c81b"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "440cf4f34310cb403ff446cc17bd6544"
server_key_iv: "4e8acf20a2ca064d5e1192d34a2bbd98"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.575038 21599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "b28f3b2633ce441988bf131bd7e7c81b"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "440cf4f34310cb403ff446cc17bd6544"
server_key_iv: "4e8acf20a2ca064d5e1192d34a2bbd98"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.578439 21599 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260621 07:57:24.580700 21613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.581790 21599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:57:24.581915 21599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "b28f3b2633ce441988bf131bd7e7c81b"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "440cf4f34310cb403ff446cc17bd6544"
server_key_iv: "4e8acf20a2ca064d5e1192d34a2bbd98"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.582019 21599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:24.607430 21599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:24.608016 21599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:24.608258 21599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:24.608826 21599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:24.609712 21599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:24.609781 21599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.609844 21599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:24.609894 21599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.618114 21599 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:32905
I20260621 07:57:24.618139 21726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:32905 every 8 connection(s)
I20260621 07:57:24.618897 21599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:57:24.627125 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.620477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:37065 (local address 127.11.223.190:42541)
0621 07:57:24.620645 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:24.620649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:24.621587 (+ 938us) server_negotiation.cc:407] Connection header received
0621 07:57:24.622851 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:24.622854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:24.622895 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:24.622982 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:24.623754 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.624284 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.625392 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.625599 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.625683 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.626666 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:24.626668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:24.626670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:24.626680 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:24.626695 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:24.626702 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:24.626755 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:24.626883 (+ 128us) server_negotiation.cc:299] Negotiation successful
0621 07:57:24.626955 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260621 07:57:24.627174 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21599
I20260621 07:57:24.627275 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:57:24.627614 12158 external_mini_cluster.cc:1428] Setting key 6e26ded9693ae16a15de6ce63d974f6e
I20260621 07:57:24.627949 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.620865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:42541 (local address 127.11.223.130:37065)
0621 07:57:24.621456 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:24.621492 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:24.622675 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:24.623124 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:24.623130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:24.623202 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:24.623636 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:24.623647 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.624437 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.624439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:24.625287 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:24.625296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.625725 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.625727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:24.625855 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.626442 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:57:24.626444 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:24.626481 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:57:24.626536 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:24.626771 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:24.626777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:24.626780 (+ 3us) client_negotiation.cc:769] Sending connection context
0621 07:57:24.626868 (+ 88us) client_negotiation.cc:240] Negotiation successful
0621 07:57:24.627077 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":104,"threads_started":1}
I20260621 07:57:24.629042 21727 heartbeater.cc:347] Connected to a master server at 127.11.223.190:42541
I20260621 07:57:24.629279 21727 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:24.629595 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42541
--builtin_ntp_servers=127.11.223.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260621 07:57:24.629766 21727 heartbeater.cc:511] Master 127.11.223.190:42541 requested a full tablet report, sending...
I20260621 07:57:24.630725 21408 ts_manager.cc:194] Registered new tserver with Master: b28f3b2633ce441988bf131bd7e7c81b (127.11.223.130:32905)
I20260621 07:57:24.631529 21408 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.130:37065
W20260621 07:57:24.734885 21730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:24.735116 21730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:24.735153 21730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:24.739143 21730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:24.739192 21730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:24.739241 21730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:57:24.743824 21730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:42541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:24.744771 21730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:24.745993 21730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:57:24.755008 21730 server_base.cc:1034] running on GCE node
W20260621 07:57:24.754974 21735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.754987 21736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.754954 21738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:24.755661 21730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:24.756181 21730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:24.757334 21730 hybrid_clock.cc:648] HybridClock initialized: now 1782028644757309 us; error 43 us; skew 500 ppm
I20260621 07:57:24.759038 21730 webserver.cc:460] Webserver started at http://127.11.223.131:40267/ using document root <none> and password file <none>
I20260621 07:57:24.759579 21730 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:24.759636 21730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:24.759810 21730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:24.761878 21730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "c1252653ebfb4de28600d2e6e5bd347d"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "c5a1710ecf47036619acd8fac77cee47"
server_key_iv: "89541d7abd949c039f0dfa714948bd7d"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.762343 21730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "c1252653ebfb4de28600d2e6e5bd347d"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "c5a1710ecf47036619acd8fac77cee47"
server_key_iv: "89541d7abd949c039f0dfa714948bd7d"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.765690 21730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260621 07:57:24.767885 21744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.769049 21730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:57:24.769176 21730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "c1252653ebfb4de28600d2e6e5bd347d"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "c5a1710ecf47036619acd8fac77cee47"
server_key_iv: "89541d7abd949c039f0dfa714948bd7d"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.769272 21730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:24.792922 21730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:24.793546 21730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:24.793730 21730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:24.794274 21730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:24.795106 21730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:24.795156 21730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.795221 21730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:24.795256 21730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:24.803582 21730 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:46455
I20260621 07:57:24.803606 21857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:46455 every 8 connection(s)
I20260621 07:57:24.804507 21730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:57:24.805938 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21730
I20260621 07:57:24.806041 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:57:24.806382 12158 external_mini_cluster.cc:1428] Setting key ef8b5b24e56d294c3386f2d0ed56c46d
I20260621 07:57:24.813073 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.806340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:53819 (local address 127.11.223.190:42541)
0621 07:57:24.806491 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:24.806502 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:24.807672 (+ 1170us) server_negotiation.cc:407] Connection header received
0621 07:57:24.809052 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:24.809055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:24.809116 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:24.809189 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:24.809976 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.810478 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.811431 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.811649 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.811721 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.812699 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:24.812702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:24.812704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:24.812714 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:24.812743 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:24.812750 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:24.812807 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:24.812932 (+ 125us) server_negotiation.cc:299] Negotiation successful
0621 07:57:24.812974 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:57:24.813807 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:24.806843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:42541 (local address 127.11.223.131:53819)
0621 07:57:24.807529 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:24.807576 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:24.808871 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:24.809336 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:24.809342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:24.809418 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:24.809862 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:24.809874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.810567 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.810569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:24.811335 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:24.811343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:24.811765 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:24.811767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:24.811893 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:24.812475 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:57:24.812477 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:24.812514 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:57:24.812568 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:24.812822 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:24.812827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:24.812830 (+ 3us) client_negotiation.cc:769] Sending connection context
0621 07:57:24.812915 (+ 85us) client_negotiation.cc:240] Negotiation successful
0621 07:57:24.813096 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":166,"threads_started":1}
I20260621 07:57:24.814791 21858 heartbeater.cc:347] Connected to a master server at 127.11.223.190:42541
I20260621 07:57:24.815028 21858 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:24.815469 21858 heartbeater.cc:511] Master 127.11.223.190:42541 requested a full tablet report, sending...
I20260621 07:57:24.816452 21408 ts_manager.cc:194] Registered new tserver with Master: c1252653ebfb4de28600d2e6e5bd347d (127.11.223.131:46455)
I20260621 07:57:24.816919 21408 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.131:53819
I20260621 07:57:24.819875 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260621 07:57:24.822602 21447 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:24.821695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.11.223.190:42541)
0621 07:57:24.821841 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:24.821845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:24.821996 (+ 151us) server_negotiation.cc:407] Connection header received
0621 07:57:24.822123 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:24.822126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:24.822171 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:24.822259 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:24.822492 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58964: BlockingRecv error: recv got EOF from 127.0.0.1:58964 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260621 07:57:24.822752 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21468
I20260621 07:57:24.827785 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21599
I20260621 07:57:24.832939 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21730
I20260621 07:57:24.837596 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21377
2026-06-21T07:57:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (852 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-21T07:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:24Z Disabled control of system clock
I20260621 07:57:24.865685 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46011
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:46011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260621 07:57:24.967921 21869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:24.968175 21869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:24.968221 21869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:24.972545 21869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:24.972604 21869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:24.972621 21869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:24.972637 21869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:24.972652 21869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:24.977216 21869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:46011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:24.978247 21869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:24.979485 21869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:24.987493 21877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.987519 21874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:24.987520 21875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:24.987690 21869 server_base.cc:1034] running on GCE node
I20260621 07:57:24.988374 21869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:24.989415 21869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:24.990568 21869 hybrid_clock.cc:648] HybridClock initialized: now 1782028644990547 us; error 38 us; skew 500 ppm
I20260621 07:57:24.992225 21869 webserver.cc:460] Webserver started at http://127.11.223.190:38663/ using document root <none> and password file <none>
I20260621 07:57:24.992785 21869 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:24.992846 21869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:24.992995 21869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:24.994830 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "823d6683e011475a95352eeef4da2f41"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "1e9ee28179dd725901c0290133fdfde0"
server_key_iv: "5f9ba91d9a270bbe214224ed6329ac1b"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.995316 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "823d6683e011475a95352eeef4da2f41"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "1e9ee28179dd725901c0290133fdfde0"
server_key_iv: "5f9ba91d9a270bbe214224ed6329ac1b"
server_key_version: "encryptionkey@0"
I20260621 07:57:24.998622 21869 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:57:25.000818 21883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.001945 21869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:57:25.002081 21869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "823d6683e011475a95352eeef4da2f41"
format_stamp: "Formatted at 2026-06-21 07:57:24 on dist-test-slave-0pn3"
server_key: "1e9ee28179dd725901c0290133fdfde0"
server_key_iv: "5f9ba91d9a270bbe214224ed6329ac1b"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.002198 21869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:25.006597 21869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:25.007169 21869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:25.007349 21869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:25.014158 21869 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:46011
I20260621 07:57:25.014163 21935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:46011 every 8 connection(s)
I20260621 07:57:25.015019 21869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:57:25.018316 21936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:25.020740 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21869
I20260621 07:57:25.020835 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:57:25.021145 12158 external_mini_cluster.cc:1428] Setting key 34b4c8ab53f758732bea032b19d7d7ca
I20260621 07:57:25.023818 21936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41: Bootstrap starting.
I20260621 07:57:25.026456 21936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:25.026455 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.022532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.11.223.190:46011)
0621 07:57:25.023119 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:25.023134 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:25.023176 (+ 42us) server_negotiation.cc:407] Connection header received
0621 07:57:25.023779 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:25.023795 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:25.024121 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:25.024461 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:25.025276 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:25.025280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:25.025291 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:25.025300 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:25.025350 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:25.025365 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:25.025464 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:25.025589 (+ 125us) server_negotiation.cc:299] Negotiation successful
0621 07:57:25.025777 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":136,"threads_started":1}
I20260621 07:57:25.027268 21936 log.cc:824] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:25.029198 21936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41: No bootstrap required, opened a new log
I20260621 07:57:25.032857 21936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823d6683e011475a95352eeef4da2f41" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46011 } }
I20260621 07:57:25.033084 21936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:25.033162 21936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 823d6683e011475a95352eeef4da2f41, State: Initialized, Role: FOLLOWER
I20260621 07:57:25.033601 21936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823d6683e011475a95352eeef4da2f41" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46011 } }
I20260621 07:57:25.033762 21936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:57:25.033849 21936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:57:25.033942 21936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:25.034761 21936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823d6683e011475a95352eeef4da2f41" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46011 } }
I20260621 07:57:25.035084 21936 leader_election.cc:302] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 823d6683e011475a95352eeef4da2f41; no voters:
I20260621 07:57:25.035400 21936 leader_election.cc:288] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:57:25.035467 21941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:25.035743 21941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [term 1 LEADER]: Becoming Leader. State: Replica: 823d6683e011475a95352eeef4da2f41, State: Running, Role: LEADER
I20260621 07:57:25.036108 21941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823d6683e011475a95352eeef4da2f41" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46011 } }
I20260621 07:57:25.036613 21936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:57:25.037844 21943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 823d6683e011475a95352eeef4da2f41. Latest consensus state: current_term: 1 leader_uuid: "823d6683e011475a95352eeef4da2f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823d6683e011475a95352eeef4da2f41" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46011 } } }
I20260621 07:57:25.037855 21942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "823d6683e011475a95352eeef4da2f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823d6683e011475a95352eeef4da2f41" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 46011 } } }
I20260621 07:57:25.037977 21943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:25.037977 21942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41 [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:25.038375 21950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:25.041062 21950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:25.046109 21950 catalog_manager.cc:1347] Generated new cluster ID: dcd6447ade234ff4950d3369ddb3d965
I20260621 07:57:25.046177 21950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:25.053556 21950 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:57:25.053839 21950 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:25.063247 21950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 823d6683e011475a95352eeef4da2f41: Generated new TSK 0
I20260621 07:57:25.063938 21950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:57:25.072551 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46011
--builtin_ntp_servers=127.11.223.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260621 07:57:25.181331 21960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:25.181550 21960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:25.181587 21960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:25.185508 21960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:25.185554 21960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:25.185632 21960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:57:25.190358 21960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:25.191244 21960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:25.192549 21960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:25.201246 21965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:25.201380 21960 server_base.cc:1034] running on GCE node
W20260621 07:57:25.201258 21968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:25.201272 21966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:25.201849 21960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:25.202355 21960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:25.203518 21960 hybrid_clock.cc:648] HybridClock initialized: now 1782028645203505 us; error 25 us; skew 500 ppm
I20260621 07:57:25.208813 21960 webserver.cc:460] Webserver started at http://127.11.223.129:34755/ using document root <none> and password file <none>
I20260621 07:57:25.209379 21960 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:25.209488 21960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:25.209695 21960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:25.211668 21960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "126c2f1cd9d74e55b7a7c97cb2038d3d"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "f041e311dc3ae9ce2ef3234c5b80e694"
server_key_iv: "88580f699c227c8ad8616f5f3034194c"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.212204 21960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "126c2f1cd9d74e55b7a7c97cb2038d3d"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "f041e311dc3ae9ce2ef3234c5b80e694"
server_key_iv: "88580f699c227c8ad8616f5f3034194c"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.215565 21960 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:57:25.217885 21974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.218981 21960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:57:25.219115 21960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "126c2f1cd9d74e55b7a7c97cb2038d3d"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "f041e311dc3ae9ce2ef3234c5b80e694"
server_key_iv: "88580f699c227c8ad8616f5f3034194c"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.219228 21960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:25.223603 21960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:25.224231 21960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:25.224424 21960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:25.224917 21960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:25.225745 21960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:25.225816 21960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.225883 21960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:25.225927 21960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.234522 21960 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:41313
I20260621 07:57:25.234543 22087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:41313 every 8 connection(s)
I20260621 07:57:25.235374 21960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:57:25.238088 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 21960
I20260621 07:57:25.238217 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:57:25.238559 12158 external_mini_cluster.cc:1428] Setting key da6bc93bf610c3e404d9096671aaccbe
I20260621 07:57:25.240989 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46011
--builtin_ntp_servers=127.11.223.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260621 07:57:25.242570 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.237498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:48035 (local address 127.11.223.190:46011)
0621 07:57:25.237654 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:25.237659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:25.238866 (+ 1207us) server_negotiation.cc:407] Connection header received
0621 07:57:25.240638 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:25.240641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:25.240679 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:25.240754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:25.242012 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:25.242017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:25.242030 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:25.242046 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:25.242071 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:25.242084 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:25.242150 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:25.242400 (+ 250us) server_negotiation.cc:299] Negotiation successful
0621 07:57:25.242442 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:57:25.243988 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.238009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46011 (local address 127.11.223.129:48035)
0621 07:57:25.238725 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:25.238773 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:25.240425 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:25.240908 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:25.240914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:25.240993 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:25.241675 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:57:25.241679 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:25.241735 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:57:25.241829 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:25.242230 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:25.242238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:25.242245 (+ 7us) client_negotiation.cc:769] Sending connection context
0621 07:57:25.242828 (+ 583us) client_negotiation.cc:240] Negotiation successful
0621 07:57:25.243097 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":162,"threads_started":1}
I20260621 07:57:25.245010 22088 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46011
I20260621 07:57:25.245209 22088 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:25.245604 22088 heartbeater.cc:511] Master 127.11.223.190:46011 requested a full tablet report, sending...
I20260621 07:57:25.247037 21900 ts_manager.cc:194] Registered new tserver with Master: 126c2f1cd9d74e55b7a7c97cb2038d3d (127.11.223.129:41313)
W20260621 07:57:25.344547 22091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:25.344758 22091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:25.344795 22091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:25.348618 22091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:25.348661 22091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:25.348711 22091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:57:25.353204 22091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:25.354027 22091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:25.355168 22091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:25.363711 22097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:25.363780 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:25.363755 22091 server_base.cc:1034] running on GCE node
W20260621 07:57:25.363719 22096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:25.364287 22091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:25.364815 22091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:25.366004 22091 hybrid_clock.cc:648] HybridClock initialized: now 1782028645365989 us; error 28 us; skew 500 ppm
I20260621 07:57:25.367698 22091 webserver.cc:460] Webserver started at http://127.11.223.130:43731/ using document root <none> and password file <none>
I20260621 07:57:25.368180 22091 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:25.368256 22091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:25.368490 22091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:25.370326 22091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "0742edf16ce348f5b2de215f3a7f47b4"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "fd4d9443b053e15ef1310002bcb09000"
server_key_iv: "0b6c2484662fe3dc225badbaa5533071"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.370844 22091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "0742edf16ce348f5b2de215f3a7f47b4"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "fd4d9443b053e15ef1310002bcb09000"
server_key_iv: "0b6c2484662fe3dc225badbaa5533071"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.374163 22091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:57:25.376204 22105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.377278 22091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:25.377388 22091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "0742edf16ce348f5b2de215f3a7f47b4"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "fd4d9443b053e15ef1310002bcb09000"
server_key_iv: "0b6c2484662fe3dc225badbaa5533071"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.377497 22091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:25.381564 22091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:25.382094 22091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:25.382263 22091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:25.382747 22091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:25.383545 22091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:25.383612 22091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.383678 22091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:25.383731 22091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.391925 22091 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:42155
I20260621 07:57:25.392004 22218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:42155 every 8 connection(s)
I20260621 07:57:25.392836 22091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:57:25.397823 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 22091
I20260621 07:57:25.397950 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:57:25.398286 12158 external_mini_cluster.cc:1428] Setting key d767be699a79cb74db1b2a28969aba2a
I20260621 07:57:25.398456 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.394413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:46403 (local address 127.11.223.190:46011)
0621 07:57:25.394555 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:25.394558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:25.395533 (+ 975us) server_negotiation.cc:407] Connection header received
0621 07:57:25.396798 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:25.396801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:25.396836 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:25.396909 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:25.398029 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:25.398033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:25.398035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:25.398048 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:25.398075 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:25.398083 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:25.398156 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:25.398313 (+ 157us) server_negotiation.cc:299] Negotiation successful
0621 07:57:25.398348 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 07:57:25.399271 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.394827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46011 (local address 127.11.223.130:46403)
0621 07:57:25.395397 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:25.395432 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:25.396578 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:25.397045 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:25.397050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:25.397126 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:25.397689 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:57:25.397691 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:25.397740 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:57:25.397824 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:25.398171 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:25.398179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:25.398185 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 07:57:25.398349 (+ 164us) client_negotiation.cc:240] Negotiation successful
0621 07:57:25.398559 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":101,"threads_started":1}
I20260621 07:57:25.400249 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46011
--builtin_ntp_servers=127.11.223.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260621 07:57:25.402119 22219 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46011
I20260621 07:57:25.402366 22219 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:25.403028 22219 heartbeater.cc:511] Master 127.11.223.190:46011 requested a full tablet report, sending...
I20260621 07:57:25.403990 21900 ts_manager.cc:194] Registered new tserver with Master: 0742edf16ce348f5b2de215f3a7f47b4 (127.11.223.130:42155)
W20260621 07:57:25.509878 22222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:25.510138 22222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:25.510217 22222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:25.514277 22222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:25.514330 22222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:25.514384 22222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:57:25.519052 22222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:25.519996 22222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:25.521349 22222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:25.530388 22230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:25.530505 22228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:25.530354 22227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:25.530614 22222 server_base.cc:1034] running on GCE node
I20260621 07:57:25.531023 22222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:25.531654 22222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:25.532829 22222 hybrid_clock.cc:648] HybridClock initialized: now 1782028645532800 us; error 43 us; skew 500 ppm
I20260621 07:57:25.534449 22222 webserver.cc:460] Webserver started at http://127.11.223.131:46089/ using document root <none> and password file <none>
I20260621 07:57:25.534976 22222 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:25.535060 22222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:25.535274 22222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:25.537271 22222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "38e591cd265947b4889e82e27d3764b6"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "c39dacd9133bf98b4fcff9471d701ce1"
server_key_iv: "8556d91a34b03c73780713f5082b340e"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.538062 22222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "38e591cd265947b4889e82e27d3764b6"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "c39dacd9133bf98b4fcff9471d701ce1"
server_key_iv: "8556d91a34b03c73780713f5082b340e"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.541663 22222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:57:25.544023 22236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.545222 22222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:25.545348 22222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "38e591cd265947b4889e82e27d3764b6"
format_stamp: "Formatted at 2026-06-21 07:57:25 on dist-test-slave-0pn3"
server_key: "c39dacd9133bf98b4fcff9471d701ce1"
server_key_iv: "8556d91a34b03c73780713f5082b340e"
server_key_version: "encryptionkey@0"
I20260621 07:57:25.545456 22222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:25.549665 22222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:25.550235 22222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:25.550406 22222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:25.550899 22222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:25.551749 22222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:25.551819 22222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.551884 22222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:25.551935 22222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:25.560274 22222 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:36493
I20260621 07:57:25.560307 22349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:36493 every 8 connection(s)
I20260621 07:57:25.561110 22222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:57:25.566665 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.562727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:57547 (local address 127.11.223.190:46011)
0621 07:57:25.562895 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:25.562900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:25.563939 (+ 1039us) server_negotiation.cc:407] Connection header received
0621 07:57:25.565240 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:25.565243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:25.565277 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:25.565352 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:25.566307 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:25.566310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:25.566311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 07:57:25.566321 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:25.566339 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 07:57:25.566346 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:25.566414 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:25.566524 (+ 110us) server_negotiation.cc:299] Negotiation successful
0621 07:57:25.566563 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 07:57:25.567010 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 22222
I20260621 07:57:25.567121 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:57:25.567427 12158 external_mini_cluster.cc:1428] Setting key e9b786f33911d3a165e5d36d375a36cb
I20260621 07:57:25.567667 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:25.563162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:46011 (local address 127.11.223.131:57547)
0621 07:57:25.563803 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:25.563842 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:25.565051 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:25.565500 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:25.565507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:25.565586 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:25.566078 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 07:57:25.566079 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:25.566114 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 07:57:25.566182 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:25.566416 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:25.566421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:25.566425 (+ 4us) client_negotiation.cc:769] Sending connection context
0621 07:57:25.566540 (+ 115us) client_negotiation.cc:240] Negotiation successful
0621 07:57:25.566736 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260621 07:57:25.568723 22350 heartbeater.cc:347] Connected to a master server at 127.11.223.190:46011
I20260621 07:57:25.568923 22350 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:25.569329 22350 heartbeater.cc:511] Master 127.11.223.190:46011 requested a full tablet report, sending...
I20260621 07:57:25.570222 21899 ts_manager.cc:194] Registered new tserver with Master: 38e591cd265947b4889e82e27d3764b6 (127.11.223.131:36493)
I20260621 07:57:25.581180 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260621 07:57:25.583669 21939 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:57:25.582964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.11.223.190:46011)
0621 07:57:25.583084 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:25.583087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:25.583263 (+ 176us) server_negotiation.cc:407] Connection header received
0621 07:57:25.583360 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:25.583362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:25.583388 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:25.583445 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:25.583590 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39798: BlockingRecv error: recv got EOF from 127.0.0.1:39798 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 07:57:25.583887 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21960
I20260621 07:57:25.589385 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 22091
I20260621 07:57:25.594389 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 22222
I20260621 07:57:25.599093 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 21869
2026-06-21T07:57:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (761 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:57:25 dist-test-slave-0pn3 krb5kdc[22360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:57:25 dist-test-slave-0pn3 krb5kdc[22360](info): set up 2 sockets
Jun 21 07:57:25 dist-test-slave-0pn3 krb5kdc[22360](info): commencing operation
krb5kdc: starting...
W20260621 07:57:26.629525 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:57:26 dist-test-slave-0pn3 krb5kdc[22360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:57:26.743121 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:41951
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:41951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:26.852658 22373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:26.852958 22373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:26.853047 22373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:26.856901 22373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:57:26.856969 22373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:26.857008 22373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:26.857060 22373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:57:26.857097 22373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:57:26.862201 22373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:41951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:41951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:26.863222 22373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:26.864224 22373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:26.872279 22379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:26.872269 22381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:26.872269 22378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:26.872424 22373 server_base.cc:1034] running on GCE node
I20260621 07:57:26.873081 22373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:26.874058 22373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:26.875219 22373 hybrid_clock.cc:648] HybridClock initialized: now 1782028646875196 us; error 34 us; skew 500 ppm
Jun 21 07:57:26 dist-test-slave-0pn3 krb5kdc[22360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:26.877777 22373 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:57:26.878605 22373 webserver.cc:460] Webserver started at http://127.11.223.190:34779/ using document root <none> and password file <none>
I20260621 07:57:26.879197 22373 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:26.879283 22373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:26.879490 22373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:26.881412 22373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "403fdbbf6cd74133a6305970930263df"
format_stamp: "Formatted at 2026-06-21 07:57:26 on dist-test-slave-0pn3"
server_key: "f09d2b8f8eb9716ecbcf70fd7dedb801"
server_key_iv: "eaf113bb38ee2b1d6251335fb29914f9"
server_key_version: "encryptionkey@0"
I20260621 07:57:26.881958 22373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "403fdbbf6cd74133a6305970930263df"
format_stamp: "Formatted at 2026-06-21 07:57:26 on dist-test-slave-0pn3"
server_key: "f09d2b8f8eb9716ecbcf70fd7dedb801"
server_key_iv: "eaf113bb38ee2b1d6251335fb29914f9"
server_key_version: "encryptionkey@0"
I20260621 07:57:26.885327 22373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260621 07:57:26.887535 22388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:26.888720 22373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:26.888844 22373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "403fdbbf6cd74133a6305970930263df"
format_stamp: "Formatted at 2026-06-21 07:57:26 on dist-test-slave-0pn3"
server_key: "f09d2b8f8eb9716ecbcf70fd7dedb801"
server_key_iv: "eaf113bb38ee2b1d6251335fb29914f9"
server_key_version: "encryptionkey@0"
I20260621 07:57:26.888943 22373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:26.901623 22373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:26.904152 22373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:26.904400 22373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:26.910977 22373 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:41951
I20260621 07:57:26.911293 22440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:41951 every 8 connection(s)
I20260621 07:57:26.911888 22373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:57:26.914698 22441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:26.918403 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 22373
I20260621 07:57:26.918503 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:57:26.918803 12158 external_mini_cluster.cc:1428] Setting key dab701a5a4935b44e1e55ad757c7922b
I20260621 07:57:26.920400 22441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: Bootstrap starting.
I20260621 07:57:26.922513 22441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:26.923228 22441 log.cc:824] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:26.925057 22441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: No bootstrap required, opened a new log
Jun 21 07:57:26 dist-test-slave-0pn3 krb5kdc[22360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:26.929179 22441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "403fdbbf6cd74133a6305970930263df" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41951 } }
I20260621 07:57:26.929415 22441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:26.929512 22441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 403fdbbf6cd74133a6305970930263df, State: Initialized, Role: FOLLOWER
I20260621 07:57:26.929989 22441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "403fdbbf6cd74133a6305970930263df" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41951 } }
I20260621 07:57:26.930161 22441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:57:26.930244 22441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:57:26.930338 22441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:26.931197 22441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "403fdbbf6cd74133a6305970930263df" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41951 } }
I20260621 07:57:26.931535 22441 leader_election.cc:302] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 403fdbbf6cd74133a6305970930263df; no voters:
I20260621 07:57:26.931874 22441 leader_election.cc:288] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:57:26.931908 22446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:26.932092 22446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [term 1 LEADER]: Becoming Leader. State: Replica: 403fdbbf6cd74133a6305970930263df, State: Running, Role: LEADER
I20260621 07:57:26.932420 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:26.920103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.11.223.190:41951)
0621 07:57:26.920576 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:26.920591 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:26.920625 (+ 34us) server_negotiation.cc:407] Connection header received
0621 07:57:26.921211 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:26.921227 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:26.921526 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:26.921842 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:26.922625 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:26.923644 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:26.924546 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:26.924986 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:26.925194 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:26.927715 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:26.927720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:26.927731 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:26.927755 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:26.929443 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:26.929879 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:26.929885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:26.929890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:26.930039 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:26.930373 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:26.930377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:26.930379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:26.930751 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:26.930970 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:26.931228 (+ 258us) server_negotiation.cc:299] Negotiation successful
0621 07:57:26.931533 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":156,"threads_started":1}
I20260621 07:57:26.932458 22446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "403fdbbf6cd74133a6305970930263df" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41951 } }
I20260621 07:57:26.933100 22441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:57:26.935493 22448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "403fdbbf6cd74133a6305970930263df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "403fdbbf6cd74133a6305970930263df" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41951 } } }
I20260621 07:57:26.935618 22448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [sys.catalog]: This master's current role is: LEADER
I20260621 07:57:26.935840 22447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 403fdbbf6cd74133a6305970930263df. Latest consensus state: current_term: 1 leader_uuid: "403fdbbf6cd74133a6305970930263df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "403fdbbf6cd74133a6305970930263df" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41951 } } }
I20260621 07:57:26.936133 22447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df [sys.catalog]: This master's current role is: LEADER
W20260621 07:57:26.938313 22460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:57:26.938400 22460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:57:26.938462 22461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:57:26.939090 22461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:57:26.943758 22461 catalog_manager.cc:1347] Generated new cluster ID: c5d25e40ea3d4898879e9c3ce957efd2
I20260621 07:57:26.943817 22461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:57:26.962172 22461 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:57:26.963367 22461 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:57:26.969480 22461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: Generated new TSK 0
I20260621 07:57:26.970242 22461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:57:26.999050 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41951
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:27.110890 22467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:27.111138 22467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:27.111182 22467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:27.115264 22467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:27.115315 22467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:27.115408 22467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:57:27.120638 22467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:27.121613 22467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:27.122579 22467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:57:27.131815 22467 server_base.cc:1034] running on GCE node
W20260621 07:57:27.131750 22473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:27.131783 22472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:27.131770 22475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:27.132516 22467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:27.133102 22467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:27.134280 22467 hybrid_clock.cc:648] HybridClock initialized: now 1782028647134252 us; error 41 us; skew 500 ppm
Jun 21 07:57:27 dist-test-slave-0pn3 krb5kdc[22360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028647, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:27.137226 22467 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:57:27.137976 22467 webserver.cc:460] Webserver started at http://127.11.223.129:43565/ using document root <none> and password file <none>
I20260621 07:57:27.138736 22467 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:27.138852 22467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:27.139106 22467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:27.141254 22467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "6ee51648f5ef457eacdd61ac90c8d78e"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "794ec367c9506c724883dfc41cb1a936"
server_key_iv: "b9a4c9c290a36a6317618a9a72e9af28"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.141810 22467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "6ee51648f5ef457eacdd61ac90c8d78e"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "794ec367c9506c724883dfc41cb1a936"
server_key_iv: "b9a4c9c290a36a6317618a9a72e9af28"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.145398 22467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260621 07:57:27.147660 22482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.148926 22467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:27.149085 22467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "6ee51648f5ef457eacdd61ac90c8d78e"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "794ec367c9506c724883dfc41cb1a936"
server_key_iv: "b9a4c9c290a36a6317618a9a72e9af28"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.149202 22467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:27.160588 22467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:27.163679 22467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:27.163866 22467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:27.164489 22467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:27.165267 22467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:27.165349 22467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.165416 22467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:27.165449 22467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.174131 22467 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:42145
I20260621 07:57:27.174154 22595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:42145 every 8 connection(s)
I20260621 07:57:27.174975 22467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
Jun 21 07:57:27 dist-test-slave-0pn3 krb5kdc[22360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028647, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:27.184827 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 22467
I20260621 07:57:27.184957 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:57:27.185235 12158 external_mini_cluster.cc:1428] Setting key 5364e94de37a465862a9f5ee369b831c
I20260621 07:57:27.187794 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.176687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:45593 (local address 127.11.223.190:41951)
0621 07:57:27.176849 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.176854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.177609 (+ 755us) server_negotiation.cc:407] Connection header received
0621 07:57:27.178847 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.178850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.178893 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.178965 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:27.179974 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.180543 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.181631 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.181846 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.181920 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.184096 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:27.184099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:27.184101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:27.184124 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:27.185716 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.186257 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.186261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.186263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.186351 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.186731 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.186734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.186735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.186905 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:27.186980 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.187571 (+ 591us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.187679 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260621 07:57:27.188917 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.177033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:41951 (local address 127.11.223.129:45593)
0621 07:57:27.177462 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.177502 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.178672 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.179090 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.179098 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.179417 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:27.179856 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.179868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.180660 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.180662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.181517 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.181525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.181966 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.181968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.182096 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.182600 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:27.182616 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:27.183959 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:27.185836 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:27.185841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:27.185854 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:27.186147 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:27.186465 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:27.186468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:27.186469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:27.186639 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:27.187013 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:27.187019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:27.187333 (+ 314us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.187538 (+ 205us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.187954 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260621 07:57:27.190176 22596 heartbeater.cc:347] Connected to a master server at 127.11.223.190:41951
I20260621 07:57:27.190418 22596 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:27.190871 22596 heartbeater.cc:511] Master 127.11.223.190:41951 requested a full tablet report, sending...
I20260621 07:57:27.192418 22405 ts_manager.cc:194] Registered new tserver with Master: 6ee51648f5ef457eacdd61ac90c8d78e (127.11.223.129:42145)
I20260621 07:57:27.193560 22405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:45593
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:57:27.209477 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41951
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:27.316517 22601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:27.316754 22601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:27.316795 22601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:27.320675 22601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:27.320722 22601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:27.320777 22601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:57:27.325654 22601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:27.326552 22601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:27.327451 22601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:27.336802 22607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:27.336809 22606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:27.336809 22609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:27.336839 22601 server_base.cc:1034] running on GCE node
I20260621 07:57:27.337497 22601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:27.338100 22601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:27.339294 22601 hybrid_clock.cc:648] HybridClock initialized: now 1782028647339279 us; error 26 us; skew 500 ppm
Jun 21 07:57:27 dist-test-slave-0pn3 krb5kdc[22360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028647, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:27.342027 22601 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:57:27.342824 22601 webserver.cc:460] Webserver started at http://127.11.223.130:36633/ using document root <none> and password file <none>
I20260621 07:57:27.343386 22601 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:27.343475 22601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:27.343704 22601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:27.345753 22601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "44bdcf79a7914c679b0fd13e611bd380"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "54bc8e69e2fbc22bc9c44bc80f83c97a"
server_key_iv: "87ca24ae645de74ac6900366de576cf5"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.346292 22601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "44bdcf79a7914c679b0fd13e611bd380"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "54bc8e69e2fbc22bc9c44bc80f83c97a"
server_key_iv: "87ca24ae645de74ac6900366de576cf5"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.349772 22601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260621 07:57:27.352149 22616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.353389 22601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:57:27.353516 22601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "44bdcf79a7914c679b0fd13e611bd380"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "54bc8e69e2fbc22bc9c44bc80f83c97a"
server_key_iv: "87ca24ae645de74ac6900366de576cf5"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.353627 22601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:27.375154 22601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:27.378311 22601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:27.378501 22601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:27.379053 22601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:27.379802 22601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:27.379868 22601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.379937 22601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:27.379987 22601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.388556 22601 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:46327
I20260621 07:57:27.388586 22729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:46327 every 8 connection(s)
I20260621 07:57:27.389398 22601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:57:27.395229 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 22601
I20260621 07:57:27.395354 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:57:27.395644 12158 external_mini_cluster.cc:1428] Setting key 7e96a443c8d1e801e3ee61e225a9e350
Jun 21 07:57:27 dist-test-slave-0pn3 krb5kdc[22360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028647, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:27.403317 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.391144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:38021 (local address 127.11.223.190:41951)
0621 07:57:27.391269 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.391273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.392066 (+ 793us) server_negotiation.cc:407] Connection header received
0621 07:57:27.393494 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.393497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.393542 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.393617 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:27.394715 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.395349 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.396775 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.397021 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.397143 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.399490 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:27.399496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:27.399500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:27.399533 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:27.401216 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.401712 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.401715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.401717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.401815 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.402124 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.402130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.402134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.402397 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:27.402508 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.402947 (+ 439us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.403087 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:57:27.403891 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.391467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:41951 (local address 127.11.223.130:38021)
0621 07:57:27.391892 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.391930 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.393311 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.393758 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.393764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.394105 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:27.394572 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.394584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.395470 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.395474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.396658 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.396668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.397172 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.397177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.397362 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.397915 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:27.397930 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:27.399338 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:27.401347 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:27.401351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:27.401361 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:27.401587 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:27.401909 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:27.401911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:27.401912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:27.402029 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:27.402508 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:27.402513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:27.402721 (+ 208us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.402861 (+ 140us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.403077 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":119,"threads_started":1}
I20260621 07:57:27.404976 22730 heartbeater.cc:347] Connected to a master server at 127.11.223.190:41951
I20260621 07:57:27.405201 22730 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:57:27.405722 22730 heartbeater.cc:511] Master 127.11.223.190:41951 requested a full tablet report, sending...
I20260621 07:57:27.406810 22405 ts_manager.cc:194] Registered new tserver with Master: 44bdcf79a7914c679b0fd13e611bd380 (127.11.223.130:46327)
I20260621 07:57:27.407344 22405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:38021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:57:27.420665 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41951
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:57:27.525761 22735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:57:27.526021 22735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:57:27.526101 22735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:57:27.530036 22735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:57:27.530117 22735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:57:27.530221 22735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:57:27.535377 22735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:57:27.536299 22735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:57:27.537240 22735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:57:27.545915 22743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:27.545931 22741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:57:27.545945 22740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:57:27.546051 22735 server_base.cc:1034] running on GCE node
I20260621 07:57:27.546672 22735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:57:27.547291 22735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:57:27.548432 22735 hybrid_clock.cc:648] HybridClock initialized: now 1782028647548414 us; error 29 us; skew 500 ppm
Jun 21 07:57:27 dist-test-slave-0pn3 krb5kdc[22360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028647, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:57:27.550972 22735 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:57:27.551810 22735 webserver.cc:460] Webserver started at http://127.11.223.131:42753/ using document root <none> and password file <none>
I20260621 07:57:27.552385 22735 fs_manager.cc:359] Metadata directory not provided
I20260621 07:57:27.552449 22735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:57:27.552662 22735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:57:27.554517 22735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "b91906d6b9394c2f8357baa0f98cf084"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "914d3c480d3e619cfc286ca2f26cb34c"
server_key_iv: "698cfe823cc6fd8a64d36c48d2fad6fa"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.555027 22735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "b91906d6b9394c2f8357baa0f98cf084"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "914d3c480d3e619cfc286ca2f26cb34c"
server_key_iv: "698cfe823cc6fd8a64d36c48d2fad6fa"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.558462 22735 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:57:27.560617 22750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.561677 22735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:57:27.561808 22735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "b91906d6b9394c2f8357baa0f98cf084"
format_stamp: "Formatted at 2026-06-21 07:57:27 on dist-test-slave-0pn3"
server_key: "914d3c480d3e619cfc286ca2f26cb34c"
server_key_iv: "698cfe823cc6fd8a64d36c48d2fad6fa"
server_key_version: "encryptionkey@0"
I20260621 07:57:27.561904 22735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:57:27.582365 22735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:57:27.584853 22735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:57:27.585031 22735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:57:27.585541 22735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:57:27.586284 22735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:57:27.586330 22735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.586395 22735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:57:27.586432 22735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:57:27.595093 22735 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:36219
I20260621 07:57:27.595129 22863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:36219 every 8 connection(s)
I20260621 07:57:27.595932 22735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
Jun 21 07:57:27 dist-test-slave-0pn3 krb5kdc[22360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028647, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:57:27.606019 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 22735
I20260621 07:57:27.606156 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:57:27.606487 12158 external_mini_cluster.cc:1428] Setting key bb67166227144bb6d6024688d8469966
I20260621 07:57:27.609064 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.597623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:34155 (local address 127.11.223.190:41951)
0621 07:57:27.597757 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.597760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.598517 (+ 757us) server_negotiation.cc:407] Connection header received
0621 07:57:27.599765 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.599768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.599814 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.599886 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:27.601019 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.601541 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.602581 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.602780 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.602853 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.605346 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:27.605349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:27.605351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:27.605374 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:27.606961 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.607452 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.607457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.607459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.607542 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.607885 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.607888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.607889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.608068 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:27.608180 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.608752 (+ 572us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.608872 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:57:27.609777 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.597968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:41951 (local address 127.11.223.131:34155)
0621 07:57:27.598365 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.598403 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.599581 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.600035 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.600042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.600428 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:57:27.600879 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.600891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.601643 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.601649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.602483 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.602491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.602898 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.602901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.603044 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.603613 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:57:27.603628 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:57:27.605187 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:57:27.607080 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:27.607086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:27.607100 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:27.607347 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:27.607647 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:57:27.607650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:57:27.607652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:57:27.607781 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:57:27.608178 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:57:27.608185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:57:27.608499 (+ 314us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.608700 (+ 201us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.609031 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260621 07:57:27.610790 22864 heartbeater.cc:347] Connected to a master server at 127.11.223.190:41951
I20260621 07:57:27.611016 22864 heartbeater.cc:464] Registering TS with master...
I20260621 07:57:27.611455 22864 heartbeater.cc:511] Master 127.11.223.190:41951 requested a full tablet report, sending...
I20260621 07:57:27.612404 22405 ts_manager.cc:194] Registered new tserver with Master: b91906d6b9394c2f8357baa0f98cf084 (127.11.223.131:36219)
I20260621 07:57:27.612962 22405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:34155
I20260621 07:57:27.619864 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:57:27.627899 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.621779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38474 (local address 127.11.223.190:41951)
0621 07:57:27.621882 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.621885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.622044 (+ 159us) server_negotiation.cc:407] Connection header received
0621 07:57:27.622142 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.622143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.622178 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.622235 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:57:27.622802 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.623273 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.624022 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.624214 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.624281 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.625531 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:57:27.625534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:57:27.625535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:57:27.625552 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:57:27.626647 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.626989 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.626991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.626992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.627057 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:57:27.627273 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:57:27.627275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:57:27.627276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:57:27.627451 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:57:27.627543 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.627717 (+ 174us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.627797 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260621 07:57:27.630621 22405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:57:27.633576 22405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:57:27.647047 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.641261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:46327 (local address 127.0.0.1:56870)
0621 07:57:27.642134 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.642182 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.642317 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.643119 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.643124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.643161 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:27.643351 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.643357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.644456 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.644460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.645662 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.645690 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.646528 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.646531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.646642 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.646678 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.646786 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.646906 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":32,"spinlock_wait_cycles":8960,"thread_start_us":72,"threads_started":1}
I20260621 07:57:27.647079 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.641681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.11.223.130:46327)
0621 07:57:27.641998 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.642013 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.642268 (+ 255us) server_negotiation.cc:407] Connection header received
0621 07:57:27.642555 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.642562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.642785 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.642935 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:27.643509 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.644268 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.645801 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.646401 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.646521 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.646651 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.646838 (+ 187us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.646927 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":112,"threads_started":1}
I20260621 07:57:27.647681 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.641399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36219 (local address 127.0.0.1:59102)
0621 07:57:27.642556 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.642568 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.642643 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.643125 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.643127 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.643163 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:27.643331 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.643338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.644856 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.644859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.646113 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.646123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.647336 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.647339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.647419 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.647438 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.647505 (+ 67us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.647566 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":81,"spinlock_wait_cycles":3072,"thread_start_us":107,"threads_started":1}
I20260621 07:57:27.647769 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.641604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.11.223.131:36219)
0621 07:57:27.642164 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.642174 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.642729 (+ 555us) server_negotiation.cc:407] Connection header received
0621 07:57:27.642805 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.642809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.642988 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.643142 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:27.643483 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.644698 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.646259 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.647026 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.647170 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.647282 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.647545 (+ 263us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.647631 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":93,"threads_started":1}
I20260621 07:57:27.647809 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.642020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:42145 (local address 127.0.0.1:34856)
0621 07:57:27.642773 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.642787 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.642897 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.643351 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.643353 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.643381 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:27.643480 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.643486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.644857 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.644859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.646185 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.646195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.647455 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.647459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.647545 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.647566 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.647631 (+ 65us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.647697 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260621 07:57:27.648116 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.642391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.11.223.129:42145)
0621 07:57:27.642770 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.642783 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.642886 (+ 103us) server_negotiation.cc:407] Connection header received
0621 07:57:27.643004 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.643008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.643211 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.643361 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:27.643550 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.644729 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.646308 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.647170 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.647280 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.647393 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.647894 (+ 501us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.647978 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":149,"threads_started":1}
I20260621 07:57:27.650044 22664 tablet_service.cc:1467] Processing CreateTablet for tablet eee0ebaf0f4c46d9a336f83f918568ab (DEFAULT_TABLE table=test-table [id=97d8782854414135bbc1d08e6978d023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:27.650352 22798 tablet_service.cc:1467] Processing CreateTablet for tablet eee0ebaf0f4c46d9a336f83f918568ab (DEFAULT_TABLE table=test-table [id=97d8782854414135bbc1d08e6978d023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:27.650651 22530 tablet_service.cc:1467] Processing CreateTablet for tablet eee0ebaf0f4c46d9a336f83f918568ab (DEFAULT_TABLE table=test-table [id=97d8782854414135bbc1d08e6978d023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:57:27.651057 22664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee0ebaf0f4c46d9a336f83f918568ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:27.651294 22798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee0ebaf0f4c46d9a336f83f918568ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:27.651487 22530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee0ebaf0f4c46d9a336f83f918568ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:57:27.657163 22881 tablet_bootstrap.cc:492] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Bootstrap starting.
I20260621 07:57:27.658571 22882 tablet_bootstrap.cc:492] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: Bootstrap starting.
I20260621 07:57:27.659861 22881 tablet_bootstrap.cc:654] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:27.660351 22882 tablet_bootstrap.cc:654] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:27.660645 22883 tablet_bootstrap.cc:492] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Bootstrap starting.
I20260621 07:57:27.660660 22881 log.cc:824] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:27.661676 22882 log.cc:824] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:27.662521 22881 tablet_bootstrap.cc:492] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: No bootstrap required, opened a new log
I20260621 07:57:27.662643 22883 tablet_bootstrap.cc:654] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Neither blocks nor log segments found. Creating new log.
I20260621 07:57:27.662747 22881 ts_tablet_manager.cc:1397] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 07:57:27.663578 22882 tablet_bootstrap.cc:492] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: No bootstrap required, opened a new log
I20260621 07:57:27.663774 22882 ts_tablet_manager.cc:1397] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:57:27.663817 22883 log.cc:824] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Log is configured to *not* fsync() on all Append() calls
I20260621 07:57:27.665724 22883 tablet_bootstrap.cc:492] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: No bootstrap required, opened a new log
I20260621 07:57:27.665896 22883 ts_tablet_manager.cc:1397] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:57:27.666234 22881 raft_consensus.cc:348] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.666455 22881 raft_consensus.cc:374] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:27.666525 22881 raft_consensus.cc:729] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bdcf79a7914c679b0fd13e611bd380, State: Initialized, Role: FOLLOWER
I20260621 07:57:27.666949 22881 consensus_queue.cc:260] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.667564 22730 heartbeater.cc:503] Master 127.11.223.190:41951 was elected leader, sending a full tablet report...
I20260621 07:57:27.667865 22881 ts_tablet_manager.cc:1428] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 07:57:27.667769 22882 raft_consensus.cc:348] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.667996 22882 raft_consensus.cc:374] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:27.668074 22882 raft_consensus.cc:729] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee51648f5ef457eacdd61ac90c8d78e, State: Initialized, Role: FOLLOWER
I20260621 07:57:27.668485 22882 consensus_queue.cc:260] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.669191 22596 heartbeater.cc:503] Master 127.11.223.190:41951 was elected leader, sending a full tablet report...
I20260621 07:57:27.669613 22883 raft_consensus.cc:348] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.669837 22883 raft_consensus.cc:374] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:57:27.669894 22883 raft_consensus.cc:729] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b91906d6b9394c2f8357baa0f98cf084, State: Initialized, Role: FOLLOWER
I20260621 07:57:27.670281 22883 consensus_queue.cc:260] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.670964 22864 heartbeater.cc:503] Master 127.11.223.190:41951 was elected leader, sending a full tablet report...
I20260621 07:57:27.671211 22883 ts_tablet_manager.cc:1428] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 07:57:27.672358 22882 ts_tablet_manager.cc:1428] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
W20260621 07:57:27.676021 22597 tablet.cc:2369] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:57:27.846969 22865 tablet.cc:2369] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:57:27.890507 22731 tablet.cc:2369] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:57:27.947240 22888 raft_consensus.cc:482] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:57:27.947448 22888 raft_consensus.cc:504] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.948659 22888 leader_election.cc:288] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44bdcf79a7914c679b0fd13e611bd380 (127.11.223.130:46327), b91906d6b9394c2f8357baa0f98cf084 (127.11.223.131:36219)
I20260621 07:57:27.953799 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.948858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:46327 (local address 127.11.223.129:57663)
0621 07:57:27.949220 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.949233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.949352 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.949632 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.949635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.949672 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:27.949923 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.949930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.950933 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.950937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.952550 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.952561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.953440 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.953443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.953514 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.953534 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.953602 (+ 68us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.953655 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":177,"spinlock_wait_cycles":35840,"thread_start_us":112,"threads_started":1}
I20260621 07:57:27.953848 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.948991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:57663 (local address 127.11.223.130:46327)
0621 07:57:27.949120 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.949124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.949316 (+ 192us) server_negotiation.cc:407] Connection header received
0621 07:57:27.949465 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.949469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.949523 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.949605 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:27.950013 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.950818 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.952676 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.953322 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.953439 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.953488 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.953633 (+ 145us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.953707 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:57:27.954042 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.949025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52433 (local address 127.11.223.131:36219)
0621 07:57:27.949180 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:27.949185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:27.949383 (+ 198us) server_negotiation.cc:407] Connection header received
0621 07:57:27.949466 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:27.949469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:27.949520 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:27.949610 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:57:27.950013 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.950828 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.952676 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.953322 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.953439 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.953488 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:27.953845 (+ 357us) server_negotiation.cc:299] Negotiation successful
0621 07:57:27.953939 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260621 07:57:27.954012 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:27.948912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36219 (local address 127.11.223.129:52433)
0621 07:57:27.949293 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:27.949305 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:57:27.949388 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:57:27.949632 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:57:27.949635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:57:27.949674 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:57:27.949901 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.949910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.950933 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.950937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.952550 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:57:27.952561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:57:27.953552 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:57:27.953561 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:57:27.953704 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:27.953727 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 07:57:27.953802 (+ 75us) client_negotiation.cc:240] Negotiation successful
0621 07:57:27.953868 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":250,"spinlock_wait_cycles":6144,"thread_start_us":115,"threads_started":1}
I20260621 07:57:27.954757 22684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ebaf0f4c46d9a336f83f918568ab" candidate_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44bdcf79a7914c679b0fd13e611bd380" is_pre_election: true
I20260621 07:57:27.955030 22684 raft_consensus.cc:2454] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ee51648f5ef457eacdd61ac90c8d78e in term 0.
I20260621 07:57:27.955005 22818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ebaf0f4c46d9a336f83f918568ab" candidate_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b91906d6b9394c2f8357baa0f98cf084" is_pre_election: true
I20260621 07:57:27.955269 22818 raft_consensus.cc:2454] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ee51648f5ef457eacdd61ac90c8d78e in term 0.
I20260621 07:57:27.955464 22486 leader_election.cc:302] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44bdcf79a7914c679b0fd13e611bd380, 6ee51648f5ef457eacdd61ac90c8d78e; no voters:
I20260621 07:57:27.955724 22888 raft_consensus.cc:2781] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:57:27.955783 22888 raft_consensus.cc:482] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:57:27.955809 22888 raft_consensus.cc:3037] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:27.956821 22888 raft_consensus.cc:504] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.957129 22888 leader_election.cc:288] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [CANDIDATE]: Term 1 election: Requested vote from peers 44bdcf79a7914c679b0fd13e611bd380 (127.11.223.130:46327), b91906d6b9394c2f8357baa0f98cf084 (127.11.223.131:36219)
I20260621 07:57:27.957502 22818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ebaf0f4c46d9a336f83f918568ab" candidate_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b91906d6b9394c2f8357baa0f98cf084"
I20260621 07:57:27.957641 22818 raft_consensus.cc:3037] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:27.958603 22818 raft_consensus.cc:2454] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee51648f5ef457eacdd61ac90c8d78e in term 1.
I20260621 07:57:27.958845 22684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ebaf0f4c46d9a336f83f918568ab" candidate_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44bdcf79a7914c679b0fd13e611bd380"
I20260621 07:57:27.958892 22483 leader_election.cc:302] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ee51648f5ef457eacdd61ac90c8d78e, b91906d6b9394c2f8357baa0f98cf084; no voters:
I20260621 07:57:27.958974 22684 raft_consensus.cc:3037] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:57:27.959044 22888 raft_consensus.cc:2781] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:57:27.959232 22888 raft_consensus.cc:686] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [term 1 LEADER]: Becoming Leader. State: Replica: 6ee51648f5ef457eacdd61ac90c8d78e, State: Running, Role: LEADER
I20260621 07:57:27.959533 22888 consensus_queue.cc:237] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } }
I20260621 07:57:27.960109 22684 raft_consensus.cc:2454] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee51648f5ef457eacdd61ac90c8d78e in term 1.
I20260621 07:57:27.962568 22404 catalog_manager.cc:5495] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ee51648f5ef457eacdd61ac90c8d78e (127.11.223.129). New cstate: current_term: 1 leader_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ee51648f5ef457eacdd61ac90c8d78e" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 42145 } health_report { overall_health: HEALTHY } } }
I20260621 07:57:28.009173 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:28.006178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.11.223.129:42145)
0621 07:57:28.006300 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:57:28.006303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:57:28.006316 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:57:28.006406 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:57:28.006408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:57:28.006456 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:57:28.006544 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:57:28.006743 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:28.007205 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:28.007948 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:57:28.008142 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:57:28.008213 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:57:28.008494 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:57:28.008849 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:57:28.008933 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 07:57:28.009025 (+ 92us) server_negotiation.cc:299] Negotiation successful
0621 07:57:28.009069 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:57:28.014364 22684 raft_consensus.cc:1261] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 1 FOLLOWER]: Refusing update from remote peer 6ee51648f5ef457eacdd61ac90c8d78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:28.014364 22818 raft_consensus.cc:1261] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 1 FOLLOWER]: Refusing update from remote peer 6ee51648f5ef457eacdd61ac90c8d78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:57:28.015066 22892 consensus_queue.cc:1035] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b91906d6b9394c2f8357baa0f98cf084" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:28.015246 22888 consensus_queue.cc:1035] T eee0ebaf0f4c46d9a336f83f918568ab P 6ee51648f5ef457eacdd61ac90c8d78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "44bdcf79a7914c679b0fd13e611bd380" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 46327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:57:28.025588 22897 mvcc.cc:205] Tried to move back new op lower bound from 7299189342260473856 to 7299189342044983296. Current Snapshot: MvccSnapshot[applied={T|T < 7299189342260473856 or (T in {7299189342260473856})}]
I20260621 07:57:28.029948 22404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:28.030110 22404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:57:28.032368 22404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 403fdbbf6cd74133a6305970930263df: Sending DeleteTablet for 3 replicas of tablet eee0ebaf0f4c46d9a336f83f918568ab
I20260621 07:57:28.033097 22798 tablet_service.cc:1514] Processing DeleteTablet for tablet eee0ebaf0f4c46d9a336f83f918568ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:28 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:59102
I20260621 07:57:28.033154 22664 tablet_service.cc:1514] Processing DeleteTablet for tablet eee0ebaf0f4c46d9a336f83f918568ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:28 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:56870
I20260621 07:57:28.033632 22905 tablet_replica.cc:330] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: stopping tablet replica
I20260621 07:57:28.033736 22530 tablet_service.cc:1514] Processing DeleteTablet for tablet eee0ebaf0f4c46d9a336f83f918568ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:57:28 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:34856
I20260621 07:57:28.033860 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 22467
I20260621 07:57:28.033995 22905 raft_consensus.cc:2229] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:28.034265 22905 raft_consensus.cc:2258] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:28.035248 22904 tablet_replica.cc:330] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: stopping tablet replica
I20260621 07:57:28.035547 22904 raft_consensus.cc:2229] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:57:28.035871 22904 raft_consensus.cc:2258] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:57:28.036051 22905 ts_tablet_manager.cc:1905] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:28.037297 22904 ts_tablet_manager.cc:1905] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:57:28.038757 22905 ts_tablet_manager.cc:1918] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:28.038858 22905 log.cc:1196] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/eee0ebaf0f4c46d9a336f83f918568ab
I20260621 07:57:28.039268 22905 ts_tablet_manager.cc:1939] T eee0ebaf0f4c46d9a336f83f918568ab P 44bdcf79a7914c679b0fd13e611bd380: Deleting consensus metadata
I20260621 07:57:28.039968 22904 ts_tablet_manager.cc:1918] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:57:28.040062 22904 log.cc:1196] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/eee0ebaf0f4c46d9a336f83f918568ab
I20260621 07:57:28.040431 22904 ts_tablet_manager.cc:1939] T eee0ebaf0f4c46d9a336f83f918568ab P b91906d6b9394c2f8357baa0f98cf084: Deleting consensus metadata
I20260621 07:57:28.040557 22392 catalog_manager.cc:4841] TS 44bdcf79a7914c679b0fd13e611bd380 (127.11.223.130:46327): tablet eee0ebaf0f4c46d9a336f83f918568ab (table test-table [id=97d8782854414135bbc1d08e6978d023]) successfully deleted
I20260621 07:57:28.041440 22389 catalog_manager.cc:4841] TS b91906d6b9394c2f8357baa0f98cf084 (127.11.223.131:36219): tablet eee0ebaf0f4c46d9a336f83f918568ab (table test-table [id=97d8782854414135bbc1d08e6978d023]) successfully deleted
W20260621 07:57:28.042146 22392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:42145 (error 108)
I20260621 07:57:28.042305 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 22601
I20260621 07:57:28.043774 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:57:28.043509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:42145 (local address 127.0.0.1:34884)
0621 07:57:28.043648 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0621 07:57:28.043704 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:42145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260621 07:57:28.043911 22392 catalog_manager.cc:4584] TS 6ee51648f5ef457eacdd61ac90c8d78e (127.11.223.129:42145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eee0ebaf0f4c46d9a336f83f918568ab: Network error: Client connection negotiation failed: client connection to 127.11.223.129:42145: connect: Connection refused (error 111)
I20260621 07:57:28.048405 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 22735
I20260621 07:57:28.053865 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 22373
2026-06-21T07:57:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2456 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:57:28 dist-test-slave-0pn3 krb5kdc[22909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:57:28 dist-test-slave-0pn3 krb5kdc[22909](info): set up 2 sockets
Jun 21 07:57:28 dist-test-slave-0pn3 krb5kdc[22909](info): commencing operation
krb5kdc: starting...
W20260621 07:57:29.065488 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:57:29 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:57:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/rangeradmin_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/rangeradmin_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/rangerlookup_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/rangerlookup_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
I20260621 07:57:29.216347 12158 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKM8XKG/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/postgres -l logfile start
2026-06-21 07:57:31.402 UTC [22939] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-21 07:57:31.402 UTC [22939] LOG: listening on IPv4 address "127.11.223.148", port 44267
2026-06-21 07:57:31.404 UTC [22939] LOG: listening on Unix socket "/tmp/.s.PGSQL.44267"
2026-06-21 07:57:31.424 UTC [22942] LOG: database system was shut down at 2026-06-21 07:57:30 UTC
2026-06-21 07:57:31.427 UTC [22939] LOG: database system is ready to accept connections
I20260621 07:57:32.361337 12158 mini_postgres.cc:96] Postgres bound to 44267
2026-06-21 07:57:32.367 UTC [22950] FATAL: database "slave" does not exist
127.11.223.148:44267 - accepting connections
I20260621 07:57:32.368564 12158 mini_ranger.cc:161] Starting Ranger...
I20260621 07:57:32.382983 12158 mini_ranger.cc:84] Created miniranger Postgres user
I20260621 07:57:32.784049 12158 mini_ranger.cc:87] Created ranger Postgres database
I20260621 07:57:32.784142 12158 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-admin
2026-06-21 07:57:33,054 [I] DB FLAVOR :POSTGRES
2026-06-21 07:57:33,054 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 07:57:33,054 [I] Checking connection..
2026-06-21 07:57:33,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-21 07:57:33,304 [I] Checking connection passed.
2026-06-21 07:57:33,304 [I] --------- Verifying version history table ---------
2026-06-21 07:57:33,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:57:33,551 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 07:57:33,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:57:33,796 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 07:57:33,796 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-21 07:57:33,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-21 07:57:34.026 UTC [23014] WARNING: there is no transaction in progress
2026-06-21 07:57:34,032 [I] create_dbversion_catalog.sql file imported successfully
2026-06-21 07:57:34,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:57:34,282 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-21 07:57:34,283 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 07:57:34,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-21 07:57:34,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-21 07:57:34,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0pn3.c.gcp-upstream.internal','N') ;"
2026-06-21 07:57:34,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 07:57:35,198 [I] Table x_portal_user does not exist in database ranger
2026-06-21 07:57:35,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 07:57:35,448 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-21 07:57:35,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 07:57:35,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 07:57:35,911 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-21 07:57:35,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-21 07:57:36.784 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.800 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.816 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.830 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.847 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.862 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.922 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.928 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.941 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.951 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.960 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.972 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.981 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.987 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.993 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:36.998 UTC [23122] WARNING: there is no transaction in progress
2026-06-21 07:57:37,536 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-21 07:57:37,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-21 07:57:37,767 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-21 07:57:37,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 07:57:38,014 [I] Table x_portal_user already exists in database 'ranger'
2026-06-21 07:57:38,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 07:57:38,272 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-21 07:57:38,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 07:57:38,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 07:57:38,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0pn3.c.gcp-upstream.internal';"
2026-06-21 07:57:38,988 [I] CORE_DB_SCHEMA import status has been updated
2026-06-21 07:57:38,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-21 07:57:39,218 [I] DB_PATCHES have already been applied
I20260621 07:57:39.223964 12158 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKM8XKG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260621 07:57:39.224041 12158 mini_ranger.cc:193] Using host: 127.11.223.148
I20260621 07:57:39.227389 12158 mini_ranger.cc:227] Ranger admin URL: http://127.11.223.148:48215
Jun 21, 2026 7:57:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 7:57:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 7:57:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp
Jun 21, 2026 7:57:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 21, 2026 7:57:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 21, 2026 7:57:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp].
Jun 21, 2026 7:57:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260621 07:57:40.290815 12158 mini_ranger.cc:160] Time spent starting Ranger: real 7.922s user 0.000s sys 0.009s
Jun 21, 2026 7:57:59 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 21, 2026 7:57:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 21, 2026 7:57:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 21, 2026 7:57:59 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 21, 2026 7:58:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260621 07:58:00.904541 12158 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/rangerkms_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/rangerkms_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.223.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.223.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 21 07:58:00 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028680, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260621 07:58:00.983492 12158 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260621 07:58:00.992180 12158 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260621 07:58:01.714547 12158 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260621 07:58:01.714648 12158 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms
I20260621 07:58:01.714751 12158 mini_ranger_kms.cc:227] Using postgres at 127.11.223.148:44267
2026-06-21 07:58:01,891 [I] DB FLAVOR :POSTGRES
2026-06-21 07:58:01,891 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 07:58:01,891 [I] Checking connection
2026-06-21 07:58:01,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-21 07:58:02,118 [I] connection success
2026-06-21 07:58:02,118 [I] --------- Verifying Ranger DB tables ---------
2026-06-21 07:58:02,119 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-21 07:58:02,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-21 07:58:02,363 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-21 07:58:02,363 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 07:58:02,363 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-21 07:58:02,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:44267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-21 07:58:02,606 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260621 07:58:03.029819 12158 mini_ranger_kms.cc:310] Created kms service
I20260621 07:58:03.223534 12158 mini_ranger_kms.cc:326] Created kudu user
I20260621 07:58:03.266717 12158 mini_ranger_kms.cc:343] Created rangerkms user
Jun 21 07:58:03 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028680, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:03.657406 12158 mini_ranger_kms.cc:384] Added ranger policy
I20260621 07:58:03.657588 12158 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260621 07:58:03.657612 12158 mini_ranger_kms.cc:240] Using host: 127.11.223.148
I20260621 07:58:03.661183 12158 mini_ranger_kms.cc:276] Ranger KMS PID: 23302
I20260621 07:58:03.661329 12158 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.223.148:47515
Jun 21, 2026 7:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 7:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 7:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 21, 2026 7:58:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 21, 2026 7:58:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-kms/ews/webapp].
Jun 21, 2026 7:58:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260621 07:58:05.868108 12158 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.885s user 0.011s sys 0.148s
I20260621 07:58:05.868252 12158 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260621 07:58:05.868392 12158 mini_ranger_kms.cc:401] 127.11.223.148:47515/kms/v1/keys
Jun 21 07:58:05 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782028685, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.223.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 21 07:58:06 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028685, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.223.148@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
Jun 21 07:58:06 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028685, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.223.148@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
Jun 21 07:58:08 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:58:08.999361 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:41267
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:41267
--ranger_config_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:47515/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:09.110635 23353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:09.110903 23353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:09.110971 23353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:09.114755 23353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:58:09.114836 23353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:09.114873 23353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:09.114930 23353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:58:09.114965 23353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:58:09.120002 23353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.223.148:47515/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:41267
--ranger_config_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:41267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:09.121143 23353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:09.122223 23353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:58:09.130641 23353 server_base.cc:1034] running on GCE node
W20260621 07:58:09.130641 23358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:09.130659 23359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:09.130633 23361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:09.131354 23353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:09.132416 23353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:09.133598 23353 hybrid_clock.cc:648] HybridClock initialized: now 1782028689133577 us; error 35 us; skew 500 ppm
Jun 21 07:58:09 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028689, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:09.139377 23353 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:58:09.140226 23353 webserver.cc:460] Webserver started at http://127.11.223.190:44795/ using document root <none> and password file <none>
I20260621 07:58:09.140815 23353 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:09.140894 23353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:09.141103 23353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:09 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028689, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:09.195351 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "e133034d33e644ac8c37b69b57f07c09"
format_stamp: "Formatted at 2026-06-21 07:58:09 on dist-test-slave-0pn3"
server_key: "c4e2d2aac8e64d6d34cf74e06a49390b"
server_key_iv: "c5e9dedaf19977a76819f7dcc1a4df6c"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:09.195987 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "e133034d33e644ac8c37b69b57f07c09"
format_stamp: "Formatted at 2026-06-21 07:58:09 on dist-test-slave-0pn3"
server_key: "c4e2d2aac8e64d6d34cf74e06a49390b"
server_key_iv: "c5e9dedaf19977a76819f7dcc1a4df6c"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:09.199855 23353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260621 07:58:09.214298 23368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:09.215788 23353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:58:09.215934 23353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "e133034d33e644ac8c37b69b57f07c09"
format_stamp: "Formatted at 2026-06-21 07:58:09 on dist-test-slave-0pn3"
server_key: "c4e2d2aac8e64d6d34cf74e06a49390b"
server_key_iv: "c5e9dedaf19977a76819f7dcc1a4df6c"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:09.216076 23353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:09.230095 23353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:09.233039 23353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:09.233243 23353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:09.241259 23353 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:41267
I20260621 07:58:09.241278 23420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:41267 every 8 connection(s)
I20260621 07:58:09.242295 23353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:58:09.245114 23421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:09.247195 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 23353
I20260621 07:58:09.247315 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:58:09.247632 12158 external_mini_cluster.cc:1428] Setting key eec8f880e2cc67471ee55eca40631321
I20260621 07:58:09.251470 23421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09: Bootstrap starting.
I20260621 07:58:09.254065 23421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:09.254878 23421 log.cc:824] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:09.257225 23421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09: No bootstrap required, opened a new log
Jun 21 07:58:09 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:09.260805 23421 raft_consensus.cc:348] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133034d33e644ac8c37b69b57f07c09" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41267 } }
I20260621 07:58:09.261123 23421 raft_consensus.cc:374] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:09.261193 23421 raft_consensus.cc:729] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e133034d33e644ac8c37b69b57f07c09, State: Initialized, Role: FOLLOWER
I20260621 07:58:09.261694 23421 consensus_queue.cc:260] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133034d33e644ac8c37b69b57f07c09" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41267 } }
I20260621 07:58:09.261929 23421 raft_consensus.cc:388] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:58:09.261997 23421 raft_consensus.cc:482] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:58:09.262117 23421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:09.263185 23421 raft_consensus.cc:504] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133034d33e644ac8c37b69b57f07c09" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41267 } }
I20260621 07:58:09.263506 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:09.249207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.11.223.190:41267)
0621 07:58:09.249673 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:09.249693 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:09.249735 (+ 42us) server_negotiation.cc:407] Connection header received
0621 07:58:09.250558 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:09.250580 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:09.250972 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:09.251322 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:09.252224 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:09.253688 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:09.254945 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:09.255702 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:09.255925 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:09.258866 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:09.258872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:09.258885 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:09.258909 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:09.260452 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:09.260857 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:09.260862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:09.260870 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:09.260970 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:09.261272 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:09.261275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:09.261276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:09.261742 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:09.261946 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:09.262578 (+ 632us) server_negotiation.cc:299] Negotiation successful
0621 07:58:09.262793 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":131,"threads_started":1}
I20260621 07:58:09.263525 23421 leader_election.cc:302] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e133034d33e644ac8c37b69b57f07c09; no voters:
I20260621 07:58:09.264756 23421 leader_election.cc:288] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:58:09.264890 23426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:58:09.265146 23426 raft_consensus.cc:686] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [term 1 LEADER]: Becoming Leader. State: Replica: e133034d33e644ac8c37b69b57f07c09, State: Running, Role: LEADER
I20260621 07:58:09.265503 23426 consensus_queue.cc:237] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133034d33e644ac8c37b69b57f07c09" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41267 } }
I20260621 07:58:09.266340 23421 sys_catalog.cc:564] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:58:09.266664 23426 sys_catalog.cc:455] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e133034d33e644ac8c37b69b57f07c09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133034d33e644ac8c37b69b57f07c09" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41267 } } }
I20260621 07:58:09.266839 23426 sys_catalog.cc:458] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [sys.catalog]: This master's current role is: LEADER
I20260621 07:58:09.267063 23426 sys_catalog.cc:455] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e133034d33e644ac8c37b69b57f07c09. Latest consensus state: current_term: 1 leader_uuid: "e133034d33e644ac8c37b69b57f07c09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133034d33e644ac8c37b69b57f07c09" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 41267 } } }
I20260621 07:58:09.267215 23426 sys_catalog.cc:458] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09 [sys.catalog]: This master's current role is: LEADER
I20260621 07:58:09.268508 23421 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260621 07:58:09.271445 23430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:58:09.275434 23430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:58:09.281850 23430 catalog_manager.cc:1347] Generated new cluster ID: 622ea9a25ead4e84905fc1165640e77a
I20260621 07:58:09.281903 23430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:58:09.326350 23430 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:58:09.327669 23430 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:58:09.342762 23430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09: Generated new TSK 0
I20260621 07:58:09.343739 23430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:58:10.013094 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:47515/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41267
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:10.144435 23466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:10.144860 23466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:10.144958 23466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:10.151022 23466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:10.151117 23466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:10.151219 23466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:58:10.157707 23466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.223.148:47515/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:10.159178 23466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:10.160491 23466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:10.172631 23473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:10.172883 23466 server_base.cc:1034] running on GCE node
W20260621 07:58:10.172678 23474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:10.172636 23476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:10.173348 23466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:10.174006 23466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:10.175192 23466 hybrid_clock.cc:648] HybridClock initialized: now 1782028690175163 us; error 48 us; skew 500 ppm
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:10.178393 23466 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:58:10.179414 23466 webserver.cc:460] Webserver started at http://127.11.223.129:44441/ using document root <none> and password file <none>
I20260621 07:58:10.180101 23466 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:10.180184 23466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:10.180492 23466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:10.203114 23466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "68ceb806d5d0425ea03fed5fae5c0a08"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "3438a4bb43111ac7b47a4a94afdc8ebd"
server_key_iv: "29b6b5702107adba94d3ca0993d8bf8a"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.205334 23466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "68ceb806d5d0425ea03fed5fae5c0a08"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "3438a4bb43111ac7b47a4a94afdc8ebd"
server_key_iv: "29b6b5702107adba94d3ca0993d8bf8a"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.212493 23466 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260621 07:58:10.239094 23485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.240959 23466 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:58:10.241178 23466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "68ceb806d5d0425ea03fed5fae5c0a08"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "3438a4bb43111ac7b47a4a94afdc8ebd"
server_key_iv: "29b6b5702107adba94d3ca0993d8bf8a"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.241420 23466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:10.266351 23466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:10.270020 23466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:10.270255 23466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:10.270926 23466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:10.271965 23466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:10.272023 23466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.272073 23466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:10.272142 23466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.285115 23466 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:34917
I20260621 07:58:10.285246 23598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:34917 every 8 connection(s)
I20260621 07:58:10.286347 23466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:58:10.291744 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 23466
I20260621 07:58:10.291889 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:58:10.292204 12158 external_mini_cluster.cc:1428] Setting key 1e128e91693b30ed9e5060be85f6a497
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:58:10.304191 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.288920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:33789 (local address 127.11.223.190:41267)
0621 07:58:10.289208 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:10.289217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:10.290093 (+ 876us) server_negotiation.cc:407] Connection header received
0621 07:58:10.291334 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:10.291338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:10.291396 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:10.291500 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:10.292893 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.293662 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.295424 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.295837 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.295922 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:10.299370 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:10.299375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:10.299378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:10.299408 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:10.301439 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:10.302064 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:10.302068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:10.302069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:10.302172 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:10.302642 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:10.302646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:10.302648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:10.302880 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:10.302964 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:10.303839 (+ 875us) server_negotiation.cc:299] Negotiation successful
0621 07:58:10.303978 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
I20260621 07:58:10.305640 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.289385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:41267 (local address 127.11.223.129:33789)
0621 07:58:10.289916 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:10.289990 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:10.291082 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:10.291706 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:10.291716 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:10.292054 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:10.292715 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:10.292731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.293827 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.293831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:10.295269 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:10.295284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.296141 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.296147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:10.296477 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:10.297206 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:10.297233 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:10.299161 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:10.301568 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:10.301574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:10.301605 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:10.301946 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:10.302305 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:10.302309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:10.302312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:10.302512 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:10.303128 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:10.303136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:10.303521 (+ 385us) client_negotiation.cc:769] Sending connection context
0621 07:58:10.304362 (+ 841us) client_negotiation.cc:240] Negotiation successful
0621 07:58:10.304748 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:58:10.307190 23599 heartbeater.cc:347] Connected to a master server at 127.11.223.190:41267
I20260621 07:58:10.307417 23599 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:10.308009 23599 heartbeater.cc:511] Master 127.11.223.190:41267 requested a full tablet report, sending...
I20260621 07:58:10.309906 23385 ts_manager.cc:194] Registered new tserver with Master: 68ceb806d5d0425ea03fed5fae5c0a08 (127.11.223.129:34917)
I20260621 07:58:10.311508 23385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:33789
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:58:10.322757 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:47515/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41267
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:10.447739 23605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:10.448168 23605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:10.448230 23605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:10.452135 23605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:10.452186 23605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:10.452271 23605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:58:10.459236 23605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.223.148:47515/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:10.460465 23605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:10.461802 23605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:10.472684 23612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:10.473207 23611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:10.473737 23614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:10.474692 23605 server_base.cc:1034] running on GCE node
I20260621 07:58:10.475184 23605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:10.475970 23605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:10.477164 23605 hybrid_clock.cc:648] HybridClock initialized: now 1782028690477135 us; error 42 us; skew 500 ppm
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:10.480681 23605 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:58:10.481755 23605 webserver.cc:460] Webserver started at http://127.11.223.130:43437/ using document root <none> and password file <none>
I20260621 07:58:10.482436 23605 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:10.482515 23605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:10.482758 23605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:10.502107 23605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "6842484ae31c4ad29916ec2667820e40"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "53003e4e94b3d4fa26b5eef92e105e8f"
server_key_iv: "72906ab37fad6539ee5507b8a97cf1c3"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.502826 23605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "6842484ae31c4ad29916ec2667820e40"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "53003e4e94b3d4fa26b5eef92e105e8f"
server_key_iv: "72906ab37fad6539ee5507b8a97cf1c3"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.509953 23605 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260621 07:58:10.531570 23621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.533351 23605 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.001s
I20260621 07:58:10.533476 23605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "6842484ae31c4ad29916ec2667820e40"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "53003e4e94b3d4fa26b5eef92e105e8f"
server_key_iv: "72906ab37fad6539ee5507b8a97cf1c3"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.533603 23605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:10.549996 23605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:10.561028 23605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:10.561342 23605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:10.562095 23605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:10.563272 23605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:10.563380 23605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.563485 23605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:10.563552 23605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.578042 23605 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:43679
I20260621 07:58:10.579309 23605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:58:10.579913 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 23605
I20260621 07:58:10.580042 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:58:10.580430 12158 external_mini_cluster.cc:1428] Setting key 792a1464be99fed00c9fc4d3043a74a5
I20260621 07:58:10.580825 23734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:43679 every 8 connection(s)
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 07:58:10.597546 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.584407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:43201 (local address 127.11.223.190:41267)
0621 07:58:10.584540 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:10.584545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:10.584923 (+ 378us) server_negotiation.cc:407] Connection header received
0621 07:58:10.585815 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:10.585818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:10.585879 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:10.585977 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:10.587381 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.588223 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.589956 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.590274 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.590347 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:10.593143 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:10.593148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:10.593150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:10.593174 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:10.595050 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:10.595751 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:10.595755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:10.595757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:10.595844 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:10.596264 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:10.596268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:10.596270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:10.596523 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:10.596603 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:10.597243 (+ 640us) server_negotiation.cc:299] Negotiation successful
0621 07:58:10.597391 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 07:58:10.599458 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.583327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:41267 (local address 127.11.223.130:43201)
0621 07:58:10.584767 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:10.584807 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:10.585609 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:10.586238 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:10.586247 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:10.586577 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:10.587200 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:10.587217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.588486 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.588489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:10.589824 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:10.589834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.590532 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.590535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:10.590726 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:10.591362 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:10.591382 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:10.592948 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:10.595213 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:10.595220 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:10.595235 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:10.595629 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:10.595955 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:10.595958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:10.595960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:10.596153 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:10.596667 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:10.596673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:10.596963 (+ 290us) client_negotiation.cc:769] Sending connection context
0621 07:58:10.598001 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0621 07:58:10.598366 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1225,"thread_start_us":135,"threads_started":1}
I20260621 07:58:10.601089 23735 heartbeater.cc:347] Connected to a master server at 127.11.223.190:41267
I20260621 07:58:10.601344 23735 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:10.601910 23735 heartbeater.cc:511] Master 127.11.223.190:41267 requested a full tablet report, sending...
I20260621 07:58:10.603057 23385 ts_manager.cc:194] Registered new tserver with Master: 6842484ae31c4ad29916ec2667820e40 (127.11.223.130:43679)
I20260621 07:58:10.603770 23385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:43201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:58:10.613230 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:47515/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41267
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:10.777436 23740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:10.777830 23740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:10.777940 23740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:10.783778 23740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:10.783924 23740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:10.784052 23740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:58:10.791518 23740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.223.148:47515/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:41267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:10.793032 23740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:10.794325 23740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:10.806782 23745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:10.807164 23740 server_base.cc:1034] running on GCE node
W20260621 07:58:10.807718 23748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:10.810233 23746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:10.811818 23740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:10.812582 23740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:10.820144 23740 hybrid_clock.cc:648] HybridClock initialized: now 1782028690820055 us; error 91 us; skew 500 ppm
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:10.824522 23740 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:58:10.825713 23740 webserver.cc:460] Webserver started at http://127.11.223.131:40267/ using document root <none> and password file <none>
I20260621 07:58:10.826440 23740 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:10.826574 23740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:10.826871 23740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:10.845134 23740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "480f3eb87db64d67a44c3feeaa47bd08"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "770314c97a5439b210fbcba68f9ade70"
server_key_iv: "e13eb65993de14e92df2b2dfaac67736"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.845966 23740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "480f3eb87db64d67a44c3feeaa47bd08"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "770314c97a5439b210fbcba68f9ade70"
server_key_iv: "e13eb65993de14e92df2b2dfaac67736"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.856806 23740 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260621 07:58:10.885090 23755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.898270 23740 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.001s sys 0.000s
I20260621 07:58:10.898444 23740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "480f3eb87db64d67a44c3feeaa47bd08"
format_stamp: "Formatted at 2026-06-21 07:58:10 on dist-test-slave-0pn3"
server_key: "770314c97a5439b210fbcba68f9ade70"
server_key_iv: "e13eb65993de14e92df2b2dfaac67736"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:10.898586 23740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:10.938800 23740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:10.942580 23740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:10.942847 23740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:10.943532 23740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:10.944659 23740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:10.944770 23740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.944871 23740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:10.944936 23740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:10.955528 23740 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:43391
I20260621 07:58:10.956749 23740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:58:10.960726 23868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:43391 every 8 connection(s)
I20260621 07:58:10.965598 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 23740
I20260621 07:58:10.965720 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:58:10.966007 12158 external_mini_cluster.cc:1428] Setting key 5d293ee3507e13983ad1e18ca5b0f45a
Jun 21 07:58:10 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:10.976877 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.962477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:57501 (local address 127.11.223.190:41267)
0621 07:58:10.962603 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:10.962607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:10.964259 (+ 1652us) server_negotiation.cc:407] Connection header received
0621 07:58:10.965493 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:10.965496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:10.965550 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:10.965623 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:10.967051 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.967797 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.969466 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.969782 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.969882 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:10.972770 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:10.972774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:10.972776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:10.972803 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:10.974513 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:10.975122 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:10.975126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:10.975128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:10.975217 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:10.975625 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:10.975629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:10.975631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:10.975848 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:10.975951 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:10.976599 (+ 648us) server_negotiation.cc:299] Negotiation successful
0621 07:58:10.976721 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:58:10.977815 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.962912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:41267 (local address 127.11.223.131:57501)
0621 07:58:10.964051 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:10.964115 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:10.965141 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:10.965924 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:10.965933 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:10.966271 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:10.966896 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:10.966914 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.967947 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.967950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:10.969335 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:10.969345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.969956 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.969960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:10.970162 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:10.970839 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:10.970859 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:10.972587 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:10.974640 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:10.974645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:10.974660 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:10.974988 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:10.975314 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:10.975317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:10.975339 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:10.975513 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:10.975971 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:10.975977 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:10.976276 (+ 299us) client_negotiation.cc:769] Sending connection context
0621 07:58:10.976523 (+ 247us) client_negotiation.cc:240] Negotiation successful
0621 07:58:10.976834 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":145,"threads_started":1}
I20260621 07:58:10.979347 23869 heartbeater.cc:347] Connected to a master server at 127.11.223.190:41267
I20260621 07:58:10.979599 23869 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:10.980149 23869 heartbeater.cc:511] Master 127.11.223.190:41267 requested a full tablet report, sending...
I20260621 07:58:10.981235 23385 ts_manager.cc:194] Registered new tserver with Master: 480f3eb87db64d67a44c3feeaa47bd08 (127.11.223.131:43391)
I20260621 07:58:10.981940 23385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:57501
I20260621 07:58:10.991976 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:58:11.004578 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:10.994372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.11.223.190:41267)
0621 07:58:10.994494 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:10.994499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:10.995146 (+ 647us) server_negotiation.cc:407] Connection header received
0621 07:58:10.995333 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:10.995336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:10.995382 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:10.995433 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:10.996442 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.997183 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.998517 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:10.998838 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:10.998919 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.000819 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:11.000823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:11.000826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:11.000852 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:11.002590 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:11.003109 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:11.003113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:11.003115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:11.003202 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:11.003545 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:11.003549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:11.003551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:11.003769 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:11.003876 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.004273 (+ 397us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.004412 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:58:11.009322 23385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:58:11.014063 23385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:58:11.043380 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.028879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:34917 (local address 127.0.0.1:51722)
0621 07:58:11.031441 (+ 2562us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:11.031452 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:11.031522 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:11.032277 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:11.032282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:11.032304 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:11.034241 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.034250 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.039536 (+ 5286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.039539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.041442 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.041453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.042801 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.042805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.042916 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.042967 (+ 51us) client_negotiation.cc:769] Sending connection context
0621 07:58:11.043087 (+ 120us) client_negotiation.cc:240] Negotiation successful
0621 07:58:11.043220 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2502,"thread_start_us":82,"threads_started":1}
I20260621 07:58:11.046586 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.025852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:43391 (local address 127.0.0.1:55210)
0621 07:58:11.031023 (+ 5171us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:11.031092 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:11.031220 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:11.034363 (+ 3143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:11.034366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:11.034384 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:11.034469 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.034475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.037728 (+ 3253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.037731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.039109 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.039120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.046155 (+ 7035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.046158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.046257 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.046280 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 07:58:11.046358 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 07:58:11.046438 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4991,"thread_start_us":113,"threads_started":1}
I20260621 07:58:11.047392 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.028554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.11.223.129:34917)
0621 07:58:11.030765 (+ 2211us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:11.030772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:11.031613 (+ 841us) server_negotiation.cc:407] Connection header received
0621 07:58:11.031699 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:11.031705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:11.031935 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:11.032095 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:11.034591 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.039412 (+ 4821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.041587 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.042458 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.042582 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.042733 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.047142 (+ 4409us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.047255 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2090,"thread_start_us":118,"threads_started":1}
I20260621 07:58:11.047892 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.026893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.11.223.131:43391)
0621 07:58:11.033622 (+ 6729us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:11.033634 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:11.033657 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:58:11.033726 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:11.033731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:11.033914 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:11.034058 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:11.036495 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.037603 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.043753 (+ 6150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.045791 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.045933 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.046086 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.047648 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.047755 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6623,"thread_start_us":102,"threads_started":1}
I20260621 07:58:11.051348 23533 tablet_service.cc:1467] Processing CreateTablet for tablet 08bb7656ef794e288be1d4b997ecc7b9 (DEFAULT_TABLE table=test-table [id=76a8acab208c4d489e07cdae34056b31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:58:11.052479 23533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08bb7656ef794e288be1d4b997ecc7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:11.052364 23803 tablet_service.cc:1467] Processing CreateTablet for tablet 08bb7656ef794e288be1d4b997ecc7b9 (DEFAULT_TABLE table=test-table [id=76a8acab208c4d489e07cdae34056b31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:58:11.053299 23803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08bb7656ef794e288be1d4b997ecc7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:11.067392 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.029554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43679 (local address 127.0.0.1:45824)
0621 07:58:11.033287 (+ 3733us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:11.033300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:11.033377 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:11.061614 (+ 28237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:11.061617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:11.061644 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:11.061768 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.061775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.064537 (+ 2762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.064541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.065817 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.065829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.066981 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.066984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.067084 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.067114 (+ 30us) client_negotiation.cc:769] Sending connection context
0621 07:58:11.067164 (+ 50us) client_negotiation.cc:240] Negotiation successful
0621 07:58:11.067241 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3673,"thread_start_us":89,"threads_started":1}
I20260621 07:58:11.068198 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.029672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.11.223.130:43679)
0621 07:58:11.049217 (+ 19545us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:11.049233 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:11.049258 (+ 25us) server_negotiation.cc:407] Connection header received
0621 07:58:11.049343 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:11.049348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:11.049538 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:11.061944 (+ 12406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:11.062112 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.064398 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.066001 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.066845 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.067710 (+ 865us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.067850 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.067900 (+ 50us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.067989 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19429,"thread_start_us":66,"threads_started":1}
I20260621 07:58:11.069497 23889 tablet_bootstrap.cc:492] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Bootstrap starting.
I20260621 07:58:11.071455 23668 tablet_service.cc:1467] Processing CreateTablet for tablet 08bb7656ef794e288be1d4b997ecc7b9 (DEFAULT_TABLE table=test-table [id=76a8acab208c4d489e07cdae34056b31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:58:11.072619 23668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08bb7656ef794e288be1d4b997ecc7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:11.073187 23888 tablet_bootstrap.cc:492] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: Bootstrap starting.
I20260621 07:58:11.075562 23888 tablet_bootstrap.cc:654] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:11.076503 23888 log.cc:824] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:11.077765 23889 tablet_bootstrap.cc:654] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:11.078681 23889 log.cc:824] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:11.080952 23889 tablet_bootstrap.cc:492] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: No bootstrap required, opened a new log
I20260621 07:58:11.081174 23889 ts_tablet_manager.cc:1397] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260621 07:58:11.086571 23889 raft_consensus.cc:348] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.087020 23889 raft_consensus.cc:374] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:11.087152 23889 raft_consensus.cc:729] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480f3eb87db64d67a44c3feeaa47bd08, State: Initialized, Role: FOLLOWER
I20260621 07:58:11.087738 23889 consensus_queue.cc:260] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.088995 23889 ts_tablet_manager.cc:1428] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260621 07:58:11.089448 23869 heartbeater.cc:503] Master 127.11.223.190:41267 was elected leader, sending a full tablet report...
I20260621 07:58:11.089839 23888 tablet_bootstrap.cc:492] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: No bootstrap required, opened a new log
I20260621 07:58:11.089911 23892 tablet_bootstrap.cc:492] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Bootstrap starting.
I20260621 07:58:11.090042 23888 ts_tablet_manager.cc:1397] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260621 07:58:11.094317 23892 tablet_bootstrap.cc:654] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:11.095247 23892 log.cc:824] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:11.106818 23892 tablet_bootstrap.cc:492] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: No bootstrap required, opened a new log
I20260621 07:58:11.107030 23892 ts_tablet_manager.cc:1397] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260621 07:58:11.107374 23888 raft_consensus.cc:348] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.107699 23888 raft_consensus.cc:374] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:11.107757 23888 raft_consensus.cc:729] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ceb806d5d0425ea03fed5fae5c0a08, State: Initialized, Role: FOLLOWER
I20260621 07:58:11.108290 23888 consensus_queue.cc:260] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.109414 23888 ts_tablet_manager.cc:1428] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: Time spent starting tablet: real 0.019s user 0.003s sys 0.004s
I20260621 07:58:11.109786 23599 heartbeater.cc:503] Master 127.11.223.190:41267 was elected leader, sending a full tablet report...
I20260621 07:58:11.110850 23892 raft_consensus.cc:348] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.112082 23892 raft_consensus.cc:374] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:11.112178 23892 raft_consensus.cc:729] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6842484ae31c4ad29916ec2667820e40, State: Initialized, Role: FOLLOWER
I20260621 07:58:11.112704 23892 consensus_queue.cc:260] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.113780 23892 ts_tablet_manager.cc:1428] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260621 07:58:11.114178 23735 heartbeater.cc:503] Master 127.11.223.190:41267 was elected leader, sending a full tablet report...
Jun 21 07:58:11 dist-test-slave-0pn3 krb5kdc[22909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028690, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
W20260621 07:58:11.211506 23870 tablet.cc:2369] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:58:11.267549 23893 raft_consensus.cc:482] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:58:11.267805 23893 raft_consensus.cc:504] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.269583 23893 leader_election.cc:288] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68ceb806d5d0425ea03fed5fae5c0a08 (127.11.223.129:34917), 6842484ae31c4ad29916ec2667820e40 (127.11.223.130:43679)
I20260621 07:58:11.275286 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.270058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43679 (local address 127.11.223.131:46369)
0621 07:58:11.270475 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:11.270490 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:11.270583 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:11.271475 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:11.271478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:11.271496 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:11.271609 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.271615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.272618 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.272621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.273882 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.273893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.274841 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.274844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.274946 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.274970 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:58:11.275041 (+ 71us) client_negotiation.cc:240] Negotiation successful
0621 07:58:11.275126 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260621 07:58:11.276635 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.270769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:46369 (local address 127.11.223.130:43679)
0621 07:58:11.271019 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:11.271023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:11.271039 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:58:11.271091 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:11.271106 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:11.271153 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:11.271227 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:11.271735 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.272494 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.274020 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.274617 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.274729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.274782 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.276442 (+ 1660us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.276520 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260621 07:58:11.277274 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08bb7656ef794e288be1d4b997ecc7b9" candidate_uuid: "480f3eb87db64d67a44c3feeaa47bd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6842484ae31c4ad29916ec2667820e40" is_pre_election: true
I20260621 07:58:11.277612 23673 raft_consensus.cc:2454] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 480f3eb87db64d67a44c3feeaa47bd08 in term 0.
I20260621 07:58:11.278668 23756 leader_election.cc:302] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 480f3eb87db64d67a44c3feeaa47bd08, 6842484ae31c4ad29916ec2667820e40; no voters:
I20260621 07:58:11.279000 23893 raft_consensus.cc:2781] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:58:11.279070 23893 raft_consensus.cc:482] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:58:11.279117 23893 raft_consensus.cc:3037] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:11.280395 23893 raft_consensus.cc:504] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.283070 23893 leader_election.cc:288] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [CANDIDATE]: Term 1 election: Requested vote from peers 68ceb806d5d0425ea03fed5fae5c0a08 (127.11.223.129:34917), 6842484ae31c4ad29916ec2667820e40 (127.11.223.130:43679)
I20260621 07:58:11.283478 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08bb7656ef794e288be1d4b997ecc7b9" candidate_uuid: "480f3eb87db64d67a44c3feeaa47bd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6842484ae31c4ad29916ec2667820e40"
I20260621 07:58:11.283607 23673 raft_consensus.cc:3037] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:11.284936 23673 raft_consensus.cc:2454] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 480f3eb87db64d67a44c3feeaa47bd08 in term 1.
I20260621 07:58:11.285338 23756 leader_election.cc:302] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 480f3eb87db64d67a44c3feeaa47bd08, 6842484ae31c4ad29916ec2667820e40; no voters:
I20260621 07:58:11.285573 23893 raft_consensus.cc:2781] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:58:11.285790 23893 raft_consensus.cc:686] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 1 LEADER]: Becoming Leader. State: Replica: 480f3eb87db64d67a44c3feeaa47bd08, State: Running, Role: LEADER
I20260621 07:58:11.286593 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.270901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:42107 (local address 127.11.223.129:34917)
0621 07:58:11.271388 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:11.271392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:11.275888 (+ 4496us) server_negotiation.cc:407] Connection header received
0621 07:58:11.275943 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:11.275946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:11.275996 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:11.276063 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:11.277461 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.278207 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.282163 (+ 3956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.282790 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.282902 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.282952 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.286415 (+ 3463us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.286495 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409}
I20260621 07:58:11.287189 23893 consensus_queue.cc:237] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } }
I20260621 07:58:11.286334 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.270128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:34917 (local address 127.11.223.131:42107)
0621 07:58:11.275441 (+ 5313us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:11.275456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:11.275536 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:11.277236 (+ 1700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:11.277239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:11.277258 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:11.277331 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.277336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.280909 (+ 3573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.280912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.282008 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:11.282018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.285945 (+ 3927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.285948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:11.286031 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.286054 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 07:58:11.286127 (+ 73us) client_negotiation.cc:240] Negotiation successful
0621 07:58:11.286191 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5254,"thread_start_us":124,"threads_started":1}
W20260621 07:58:11.287889 23600 tablet.cc:2369] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:58:11.288635 23540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08bb7656ef794e288be1d4b997ecc7b9" candidate_uuid: "480f3eb87db64d67a44c3feeaa47bd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" is_pre_election: true
I20260621 07:58:11.288898 23540 raft_consensus.cc:2454] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 480f3eb87db64d67a44c3feeaa47bd08 in term 0.
I20260621 07:58:11.289275 23553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08bb7656ef794e288be1d4b997ecc7b9" candidate_uuid: "480f3eb87db64d67a44c3feeaa47bd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68ceb806d5d0425ea03fed5fae5c0a08"
I20260621 07:58:11.289383 23553 raft_consensus.cc:3037] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:11.290539 23553 raft_consensus.cc:2454] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 480f3eb87db64d67a44c3feeaa47bd08 in term 1.
I20260621 07:58:11.291630 23385 catalog_manager.cc:5495] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 480f3eb87db64d67a44c3feeaa47bd08 (127.11.223.131). New cstate: current_term: 1 leader_uuid: "480f3eb87db64d67a44c3feeaa47bd08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "480f3eb87db64d67a44c3feeaa47bd08" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 43391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 } health_report { overall_health: UNKNOWN } } }
W20260621 07:58:11.332208 23736 tablet.cc:2369] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:58:11.344167 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.339088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.11.223.131:43391)
0621 07:58:11.339230 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:11.339234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:11.339248 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:58:11.339338 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:11.339341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:11.339448 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:11.339600 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:58:11.340027 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.340850 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.342118 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:11.342444 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:11.342546 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:11.342958 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:58:11.343380 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:58:11.343520 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:11.343933 (+ 413us) server_negotiation.cc:299] Negotiation successful
0621 07:58:11.343993 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 07:58:11.351799 23673 raft_consensus.cc:1261] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 1 FOLLOWER]: Refusing update from remote peer 480f3eb87db64d67a44c3feeaa47bd08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:58:11.352039 23553 raft_consensus.cc:1261] T 08bb7656ef794e288be1d4b997ecc7b9 P 68ceb806d5d0425ea03fed5fae5c0a08 [term 1 FOLLOWER]: Refusing update from remote peer 480f3eb87db64d67a44c3feeaa47bd08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:58:11.352608 23900 consensus_queue.cc:1035] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6842484ae31c4ad29916ec2667820e40" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:58:11.352779 23893 consensus_queue.cc:1035] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ceb806d5d0425ea03fed5fae5c0a08" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:58:11.364461 23904 mvcc.cc:205] Tried to move back new op lower bound from 7299189519768031232 to 7299189519518474240. Current Snapshot: MvccSnapshot[applied={T|T < 7299189519768031232}]
I20260621 07:58:11.366883 23903 mvcc.cc:205] Tried to move back new op lower bound from 7299189519768031232 to 7299189519518474240. Current Snapshot: MvccSnapshot[applied={T|T < 7299189519768031232 or (T in {7299189519768031232})}]
I20260621 07:58:11.371480 23905 mvcc.cc:205] Tried to move back new op lower bound from 7299189519768031232 to 7299189519518474240. Current Snapshot: MvccSnapshot[applied={T|T < 7299189519768031232}]
I20260621 07:58:11.373306 23385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:58:11.373471 23385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:58:11.376384 23385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e133034d33e644ac8c37b69b57f07c09: Sending DeleteTablet for 3 replicas of tablet 08bb7656ef794e288be1d4b997ecc7b9
I20260621 07:58:11.377532 23803 tablet_service.cc:1514] Processing DeleteTablet for tablet 08bb7656ef794e288be1d4b997ecc7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:58:11 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:55210
I20260621 07:58:11.377990 23668 tablet_service.cc:1514] Processing DeleteTablet for tablet 08bb7656ef794e288be1d4b997ecc7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:58:11 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:45824
I20260621 07:58:11.379032 23533 tablet_service.cc:1514] Processing DeleteTablet for tablet 08bb7656ef794e288be1d4b997ecc7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:58:11 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:51722
I20260621 07:58:11.379516 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 23466
I20260621 07:58:11.379724 23912 tablet_replica.cc:330] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: stopping tablet replica
I20260621 07:58:11.380537 23913 tablet_replica.cc:330] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: stopping tablet replica
I20260621 07:58:11.380826 23913 raft_consensus.cc:2229] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:58:11.381120 23913 raft_consensus.cc:2258] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:58:11.381484 23912 raft_consensus.cc:2229] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:58:11.381832 23912 raft_consensus.cc:2258] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:58:11.384949 23913 ts_tablet_manager.cc:1905] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:58:11.386113 23912 ts_tablet_manager.cc:1905] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:58:11.393510 23912 ts_tablet_manager.cc:1918] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:58:11.396379 23913 ts_tablet_manager.cc:1918] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:58:11.396562 23913 log.cc:1196] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/08bb7656ef794e288be1d4b997ecc7b9
I20260621 07:58:11.396993 23913 ts_tablet_manager.cc:1939] T 08bb7656ef794e288be1d4b997ecc7b9 P 6842484ae31c4ad29916ec2667820e40: Deleting consensus metadata
I20260621 07:58:11.397154 23912 log.cc:1196] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/08bb7656ef794e288be1d4b997ecc7b9
I20260621 07:58:11.397569 23912 ts_tablet_manager.cc:1939] T 08bb7656ef794e288be1d4b997ecc7b9 P 480f3eb87db64d67a44c3feeaa47bd08: Deleting consensus metadata
I20260621 07:58:11.399151 23369 catalog_manager.cc:4841] TS 6842484ae31c4ad29916ec2667820e40 (127.11.223.130:43679): tablet 08bb7656ef794e288be1d4b997ecc7b9 (table test-table [id=76a8acab208c4d489e07cdae34056b31]) successfully deleted
W20260621 07:58:11.399756 23372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:34917 (error 108)
I20260621 07:58:11.401435 23371 catalog_manager.cc:4841] TS 480f3eb87db64d67a44c3feeaa47bd08 (127.11.223.131:43391): tablet 08bb7656ef794e288be1d4b997ecc7b9 (table test-table [id=76a8acab208c4d489e07cdae34056b31]) successfully deleted
I20260621 07:58:11.402155 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 23605
I20260621 07:58:11.405174 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:11.404844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:34917 (local address 127.0.0.1:51730)
0621 07:58:11.405017 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:11.405084 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:34917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260621 07:58:11.405336 23372 catalog_manager.cc:4584] TS 68ceb806d5d0425ea03fed5fae5c0a08 (127.11.223.129:34917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08bb7656ef794e288be1d4b997ecc7b9: Network error: Client connection negotiation failed: client connection to 127.11.223.129:34917: connect: Connection refused (error 111)
I20260621 07:58:11.411707 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 23740
I20260621 07:58:11.417868 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 23353
I20260621 07:58:11.424258 12158 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260621 07:58:11.759435 12158 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260621 07:58:11.759559 12158 mini_ranger.cc:66] Stopping Ranger...
I20260621 07:58:12.105441 12158 mini_ranger.cc:68] Stopped Ranger
2026-06-21 07:58:12.105 UTC [22939] LOG: received smart shutdown request
2026-06-21 07:58:12.108 UTC [22939] LOG: background worker "logical replication launcher" (PID 22948) exited with exit code 1
2026-06-21 07:58:12.109 UTC [22943] LOG: shutting down
2026-06-21 07:58:12.252 UTC [22939] LOG: database system is shut down
2026-06-21T07:58:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44311 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:58:12 dist-test-slave-0pn3 krb5kdc[23924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:58:12 dist-test-slave-0pn3 krb5kdc[23924](info): set up 2 sockets
Jun 21 07:58:12 dist-test-slave-0pn3 krb5kdc[23924](info): commencing operation
krb5kdc: starting...
W20260621 07:58:13.380390 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:58:13 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:58:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/rangeradmin_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/rangeradmin_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/rangerlookup_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/rangerlookup_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
I20260621 07:58:13.535097 12158 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKM8XKG/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/postgres -l logfile start
2026-06-21 07:58:15.772 UTC [23954] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-21 07:58:15.772 UTC [23954] LOG: listening on IPv4 address "127.11.223.148", port 59833
2026-06-21 07:58:15.775 UTC [23954] LOG: listening on Unix socket "/tmp/.s.PGSQL.59833"
2026-06-21 07:58:15.810 UTC [23957] LOG: database system was shut down at 2026-06-21 07:58:14 UTC
2026-06-21 07:58:15.814 UTC [23954] LOG: database system is ready to accept connections
I20260621 07:58:16.732306 12158 mini_postgres.cc:96] Postgres bound to 59833
2026-06-21 07:58:16.738 UTC [23965] FATAL: database "slave" does not exist
127.11.223.148:59833 - accepting connections
I20260621 07:58:16.738945 12158 mini_ranger.cc:161] Starting Ranger...
I20260621 07:58:16.753675 12158 mini_ranger.cc:84] Created miniranger Postgres user
I20260621 07:58:17.164930 12158 mini_ranger.cc:87] Created ranger Postgres database
I20260621 07:58:17.164996 12158 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-admin
2026-06-21 07:58:17,435 [I] DB FLAVOR :POSTGRES
2026-06-21 07:58:17,436 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 07:58:17,436 [I] Checking connection..
2026-06-21 07:58:17,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-21 07:58:17,663 [I] Checking connection passed.
2026-06-21 07:58:17,664 [I] --------- Verifying version history table ---------
2026-06-21 07:58:17,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:58:17,889 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 07:58:17,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:58:18,138 [I] Table x_db_version_h does not exist in database ranger
2026-06-21 07:58:18,138 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-21 07:58:18,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-21 07:58:18.367 UTC [24029] WARNING: there is no transaction in progress
2026-06-21 07:58:18,373 [I] create_dbversion_catalog.sql file imported successfully
2026-06-21 07:58:18,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-21 07:58:18,626 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-21 07:58:18,626 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 07:58:18,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-21 07:58:18,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-21 07:58:19,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0pn3.c.gcp-upstream.internal','N') ;"
2026-06-21 07:58:19,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 07:58:19,605 [I] Table x_portal_user does not exist in database ranger
2026-06-21 07:58:19,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 07:58:19,844 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-21 07:58:19,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 07:58:20,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 07:58:20,324 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-21 07:58:20,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-21 07:58:21.238 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.254 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.269 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.285 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.300 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.320 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.383 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.390 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.403 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.413 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.422 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.436 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.445 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.451 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.457 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:21.463 UTC [24137] WARNING: there is no transaction in progress
2026-06-21 07:58:22,063 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-21 07:58:22,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-21 07:58:22,282 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-21 07:58:22,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-21 07:58:22,524 [I] Table x_portal_user already exists in database 'ranger'
2026-06-21 07:58:22,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-21 07:58:22,782 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-21 07:58:22,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-21 07:58:23,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-21 07:58:23,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0pn3.c.gcp-upstream.internal';"
2026-06-21 07:58:23,475 [I] CORE_DB_SCHEMA import status has been updated
2026-06-21 07:58:23,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-21 07:58:23,728 [I] DB_PATCHES have already been applied
I20260621 07:58:23.734452 12158 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKM8XKG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260621 07:58:23.734519 12158 mini_ranger.cc:193] Using host: 127.11.223.148
I20260621 07:58:23.738041 12158 mini_ranger.cc:227] Ranger admin URL: http://127.11.223.148:45633
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-admin/ews/webapp].
Jun 21, 2026 7:58:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260621 07:58:24.823755 12158 mini_ranger.cc:160] Time spent starting Ranger: real 8.085s user 0.004s sys 0.005s
Jun 21, 2026 7:58:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 21, 2026 7:58:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 21, 2026 7:58:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 21, 2026 7:58:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 21, 2026 7:58:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260621 07:58:45.199244 12158 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.223.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.223.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/rangerkms_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.223.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/rangerkms_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.223.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.223.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.223.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/HTTP_127.11.223.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 21 07:58:45 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028725, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260621 07:58:45.294548 12158 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260621 07:58:45.304461 12158 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260621 07:58:46.090914 12158 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260621 07:58:46.090988 12158 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms
I20260621 07:58:46.091007 12158 mini_ranger_kms.cc:227] Using postgres at 127.11.223.148:59833
2026-06-21 07:58:46,261 [I] DB FLAVOR :POSTGRES
2026-06-21 07:58:46,261 [I] --------- Verifying Ranger DB connection ---------
2026-06-21 07:58:46,261 [I] Checking connection
2026-06-21 07:58:46,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-21 07:58:46,489 [I] connection success
2026-06-21 07:58:46,490 [I] --------- Verifying Ranger DB tables ---------
2026-06-21 07:58:46,490 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-21 07:58:46,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-21 07:58:46,715 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-21 07:58:46,716 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-21 07:58:46,716 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-21 07:58:46,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.223.148:59833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-21 07:58:46,962 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260621 07:58:47.400486 12158 mini_ranger_kms.cc:310] Created kms service
I20260621 07:58:47.580390 12158 mini_ranger_kms.cc:326] Created kudu user
I20260621 07:58:47.622752 12158 mini_ranger_kms.cc:343] Created rangerkms user
Jun 21 07:58:47 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028725, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:48.004449 12158 mini_ranger_kms.cc:384] Added ranger policy
I20260621 07:58:48.004709 12158 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKM8XKG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKM8XKG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260621 07:58:48.004737 12158 mini_ranger_kms.cc:240] Using host: 127.11.223.148
I20260621 07:58:48.008280 12158 mini_ranger_kms.cc:276] Ranger KMS PID: 24317
I20260621 07:58:48.008383 12158 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.223.148:56009
Jun 21, 2026 7:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 21, 2026 7:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 21, 2026 7:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 21, 2026 7:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 21, 2026 7:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-kms/ews/webapp].
Jun 21, 2026 7:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260621 07:58:50.236055 12158 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.942s user 0.015s sys 0.139s
I20260621 07:58:50.236194 12158 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260621 07:58:50.236270 12158 mini_ranger_kms.cc:401] 127.11.223.148:56009/kms/v1/keys
Jun 21 07:58:50 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782028730, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.223.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 21 07:58:50 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028730, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.223.148@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
Jun 21 07:58:51 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028730, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.223.148@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
Jun 21 07:58:53 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:58:53.492794 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:38229
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:38229
--ranger_config_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:53.601332 24368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:53.601593 24368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:53.601630 24368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:53.605520 24368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:58:53.605577 24368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:53.605595 24368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:53.605639 24368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:58:53.605669 24368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:58:53.610958 24368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.223.148:56009/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:38229
--ranger_config_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:38229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:53.611969 24368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:53.612993 24368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:53.621419 24374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:53.621425 24373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:53.621430 24368 server_base.cc:1034] running on GCE node
W20260621 07:58:53.621712 24376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:53.622319 24368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:53.623490 24368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:53.624666 24368 hybrid_clock.cc:648] HybridClock initialized: now 1782028733624642 us; error 45 us; skew 500 ppm
Jun 21 07:58:53 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:53.627338 24368 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:58:53.628189 24368 webserver.cc:460] Webserver started at http://127.11.223.190:43847/ using document root <none> and password file <none>
I20260621 07:58:53.628779 24368 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:53.628870 24368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:53.629082 24368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:53 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:53.684738 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "78186cb46992438a804a290656a937ad"
format_stamp: "Formatted at 2026-06-21 07:58:53 on dist-test-slave-0pn3"
server_key: "d0e65c96be33bf9519881d057b69e7bf"
server_key_iv: "451b166f7fb991064b0168cf44084488"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:53.685489 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "78186cb46992438a804a290656a937ad"
format_stamp: "Formatted at 2026-06-21 07:58:53 on dist-test-slave-0pn3"
server_key: "d0e65c96be33bf9519881d057b69e7bf"
server_key_iv: "451b166f7fb991064b0168cf44084488"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:53.690493 24368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260621 07:58:53.707770 24383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:53.709443 24368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:58:53.709599 24368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "78186cb46992438a804a290656a937ad"
format_stamp: "Formatted at 2026-06-21 07:58:53 on dist-test-slave-0pn3"
server_key: "d0e65c96be33bf9519881d057b69e7bf"
server_key_iv: "451b166f7fb991064b0168cf44084488"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:53.709734 24368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:53.739702 24368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:53.742925 24368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:53.743430 24368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:53.751900 24368 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:38229
I20260621 07:58:53.751919 24435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:38229 every 8 connection(s)
I20260621 07:58:53.753110 24368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:58:53.755916 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:53.760569 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 24368
I20260621 07:58:53.760699 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:58:53.761009 12158 external_mini_cluster.cc:1428] Setting key facc76bc941995bf33a2372f5143cd95
I20260621 07:58:53.761718 24436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Bootstrap starting.
I20260621 07:58:53.764963 24436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:53.765918 24436 log.cc:824] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:53.767901 24436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: No bootstrap required, opened a new log
Jun 21 07:58:53 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:53.772104 24436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:53.772367 24436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:53.772446 24436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78186cb46992438a804a290656a937ad, State: Initialized, Role: FOLLOWER
I20260621 07:58:53.772950 24436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:53.773128 24436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:58:53.773231 24436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:58:53.773351 24436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:53.774272 24436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:53.774612 24436 leader_election.cc:302] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78186cb46992438a804a290656a937ad; no voters:
I20260621 07:58:53.774983 24436 leader_election.cc:288] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:58:53.775070 24441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:58:53.775352 24441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 LEADER]: Becoming Leader. State: Replica: 78186cb46992438a804a290656a937ad, State: Running, Role: LEADER
I20260621 07:58:53.775709 24441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:53.776438 24436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:58:53.777252 24441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78186cb46992438a804a290656a937ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } } }
I20260621 07:58:53.777441 24441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: This master's current role is: LEADER
I20260621 07:58:53.777689 24441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78186cb46992438a804a290656a937ad. Latest consensus state: current_term: 1 leader_uuid: "78186cb46992438a804a290656a937ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } } }
I20260621 07:58:53.777854 24441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: This master's current role is: LEADER
I20260621 07:58:53.778147 24450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:58:53.778467 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:53.762474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.11.223.190:38229)
0621 07:58:53.762907 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:53.762926 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:53.762971 (+ 45us) server_negotiation.cc:407] Connection header received
0621 07:58:53.763848 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:53.763873 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:53.764376 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:53.764771 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:53.765693 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:53.767237 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:53.768517 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:53.769242 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:53.769402 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:53.772483 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:53.772489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:53.772506 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:53.772540 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:53.774558 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:53.775103 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:53.775109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:53.775115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:53.775242 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:53.775567 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:53.775570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:53.775572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:53.776077 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:53.776921 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:53.777063 (+ 142us) server_negotiation.cc:299] Negotiation successful
0621 07:58:53.777358 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":107,"threads_started":1}
I20260621 07:58:53.782047 24450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:58:53.782503 24436 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260621 07:58:53.792649 24450 catalog_manager.cc:1347] Generated new cluster ID: a47dc5a19a2045179399ab333d93ea5a
I20260621 07:58:53.792727 24450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:58:53.801427 24450 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:58:53.802397 24450 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:58:53.808128 24450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Generated new TSK 0
I20260621 07:58:53.808833 24450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:58:54.509178 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38229
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:54.651090 24481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:54.651525 24481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:54.651609 24481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:54.657694 24481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:54.657773 24481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:54.657858 24481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:58:54.665642 24481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.223.148:56009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:54.667054 24481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:54.668308 24481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:54.679862 24488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:54.680346 24491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:54.680616 24489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:54.681567 24481 server_base.cc:1034] running on GCE node
I20260621 07:58:54.682096 24481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:54.682891 24481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:54.690291 24481 hybrid_clock.cc:648] HybridClock initialized: now 1782028734690216 us; error 98 us; skew 500 ppm
Jun 21 07:58:54 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:54.694216 24481 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:58:54.695417 24481 webserver.cc:460] Webserver started at http://127.11.223.129:44915/ using document root <none> and password file <none>
I20260621 07:58:54.696180 24481 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:54.696336 24481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:54.696624 24481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:54 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:54.719635 24481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "e42038c9e1bf4700919763b5d4c1d9b7"
format_stamp: "Formatted at 2026-06-21 07:58:54 on dist-test-slave-0pn3"
server_key: "f1e4e4b3c189a142d016b85192d35427"
server_key_iv: "b0a4abb16e21e1a2609adfeb185823f4"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:54.720476 24481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "e42038c9e1bf4700919763b5d4c1d9b7"
format_stamp: "Formatted at 2026-06-21 07:58:54 on dist-test-slave-0pn3"
server_key: "f1e4e4b3c189a142d016b85192d35427"
server_key_iv: "b0a4abb16e21e1a2609adfeb185823f4"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:54.738201 24481 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.004s sys 0.001s
I20260621 07:58:54.775585 24500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:54.777397 24481 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260621 07:58:54.777530 24481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "e42038c9e1bf4700919763b5d4c1d9b7"
format_stamp: "Formatted at 2026-06-21 07:58:54 on dist-test-slave-0pn3"
server_key: "f1e4e4b3c189a142d016b85192d35427"
server_key_iv: "b0a4abb16e21e1a2609adfeb185823f4"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:54.777709 24481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Jun 21 07:58:54 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:54.823378 24481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:54.827366 24481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:54.827643 24481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:54.828441 24481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:54.829569 24481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:54.829677 24481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:54.829813 24481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:54.829900 24481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:54.840746 24481 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:39573
I20260621 07:58:54.842061 24481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:58:54.852005 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 24481
I20260621 07:58:54.852134 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:58:54.852474 12158 external_mini_cluster.cc:1428] Setting key dbcece99eba38b68fa3c927bb8f97e0d
I20260621 07:58:54.852680 24613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:39573 every 8 connection(s)
Jun 21 07:58:54 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:58:54.876538 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:54.856579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:39867 (local address 127.11.223.190:38229)
0621 07:58:54.856875 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:54.856880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:54.857338 (+ 458us) server_negotiation.cc:407] Connection header received
0621 07:58:54.859982 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:54.859986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:54.860042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:54.860113 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:54.861799 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:54.862714 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:54.864430 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:54.864715 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:54.864786 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:54.872067 (+ 7281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:54.872071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:54.872086 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:54.872119 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:54.874025 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:54.874689 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:54.874693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:54.874695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:54.874785 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:54.875213 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:54.875216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:54.875218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:54.875443 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:54.875526 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:54.876197 (+ 671us) server_negotiation.cc:299] Negotiation successful
0621 07:58:54.876353 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":100,"threads_started":1}
I20260621 07:58:54.878562 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:54.855406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38229 (local address 127.11.223.129:39867)
0621 07:58:54.857177 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:54.857234 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:54.858369 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:54.860572 (+ 2203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:54.860582 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:54.860939 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:54.861633 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:54.861652 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:54.862873 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:54.862876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:54.864280 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:54.864290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:54.864984 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:54.864987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:54.865186 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:54.865881 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:54.865902 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:54.867723 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:54.874185 (+ 6462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:54.874190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:54.874210 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:54.874574 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:54.874903 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:54.874906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:54.874908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:54.875105 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:54.875591 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:54.875597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:54.875914 (+ 317us) client_negotiation.cc:769] Sending connection context
0621 07:58:54.876998 (+ 1084us) client_negotiation.cc:240] Negotiation successful
0621 07:58:54.877363 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1533,"thread_start_us":149,"threads_started":1}
I20260621 07:58:54.880318 24614 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38229
I20260621 07:58:54.880595 24614 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:54.881174 24614 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:54.882998 24400 ts_manager.cc:194] Registered new tserver with Master: e42038c9e1bf4700919763b5d4c1d9b7 (127.11.223.129:39573)
I20260621 07:58:54.884598 24400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:39867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:58:54.907598 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38229
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:55.059957 24621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:55.060289 24621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:55.060388 24621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:55.064342 24621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:55.064430 24621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:55.064531 24621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:58:55.069921 24621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.223.148:56009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:55.071271 24621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:55.072583 24621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:55.085067 24627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:55.086369 24621 server_base.cc:1034] running on GCE node
W20260621 07:58:55.086958 24626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:55.088178 24629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:55.088737 24621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:55.089458 24621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:55.090662 24621 hybrid_clock.cc:648] HybridClock initialized: now 1782028735090625 us; error 46 us; skew 500 ppm
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:55.095652 24621 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:58:55.097295 24621 webserver.cc:460] Webserver started at http://127.11.223.130:40643/ using document root <none> and password file <none>
I20260621 07:58:55.098330 24621 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:55.098469 24621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:55.098836 24621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:55.117599 24621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "6208150410dd4bd997a72c17c68ae53b"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "8f68bfdc20871140eb7bdbeb3b29d062"
server_key_iv: "9c48cd155416720a0ad07ed5377508a9"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.118477 24621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "6208150410dd4bd997a72c17c68ae53b"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "8f68bfdc20871140eb7bdbeb3b29d062"
server_key_iv: "9c48cd155416720a0ad07ed5377508a9"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.125728 24621 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260621 07:58:55.145457 24636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.147239 24621 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260621 07:58:55.147395 24621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "6208150410dd4bd997a72c17c68ae53b"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "8f68bfdc20871140eb7bdbeb3b29d062"
server_key_iv: "9c48cd155416720a0ad07ed5377508a9"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.147558 24621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:55.178922 24621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:55.186964 24621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:55.187235 24621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:55.187902 24621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:55.189055 24621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:55.189141 24621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.189214 24621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:55.189247 24621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.202997 24621 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:43439
I20260621 07:58:55.204244 24621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:58:55.205155 24749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:43439 every 8 connection(s)
I20260621 07:58:55.212008 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 24621
I20260621 07:58:55.212138 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:58:55.212492 12158 external_mini_cluster.cc:1428] Setting key a54295f60aad3b6ac151f1c11103fa48
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:55.225921 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:55.208507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:47025 (local address 127.11.223.190:38229)
0621 07:58:55.208654 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:55.208659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:55.210245 (+ 1586us) server_negotiation.cc:407] Connection header received
0621 07:58:55.211534 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:55.211538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:55.211595 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:55.211666 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:55.214445 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.215305 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.217125 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.217503 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.217583 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:55.221156 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:55.221159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:55.221162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:55.221191 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:55.223120 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:55.223812 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:55.223815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:55.223817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:55.223900 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:55.224289 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:55.224292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:55.224294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:55.224541 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:55.224635 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:55.225589 (+ 954us) server_negotiation.cc:299] Negotiation successful
0621 07:58:55.225738 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 07:58:55.227938 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:55.208166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38229 (local address 127.11.223.130:47025)
0621 07:58:55.210071 (+ 1905us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:55.210131 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:55.211280 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:55.212095 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:55.212104 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:55.212504 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:55.214280 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:55.214302 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.215496 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.215499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:55.216982 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:55.216993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.217819 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.217823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:55.218055 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:55.218742 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:55.218764 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:55.220953 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:55.223290 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:55.223296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:55.223313 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:55.223695 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:55.224001 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:55.224004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:55.224005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:55.224183 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:55.224949 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:55.224958 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:55.225301 (+ 343us) client_negotiation.cc:769] Sending connection context
0621 07:58:55.226415 (+ 1114us) client_negotiation.cc:240] Negotiation successful
0621 07:58:55.226780 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1684,"thread_start_us":540,"threads_started":1}
I20260621 07:58:55.230921 24750 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38229
I20260621 07:58:55.231190 24750 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:55.231797 24750 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:55.233031 24400 ts_manager.cc:194] Registered new tserver with Master: 6208150410dd4bd997a72c17c68ae53b (127.11.223.130:43439)
I20260621 07:58:55.233799 24400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:47025
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:58:55.264983 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38229
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
W20260621 07:58:55.466270 24755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:55.466698 24755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:55.466778 24755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:55.472925 24755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:55.473026 24755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:55.473141 24755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:58:55.480690 24755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.223.148:56009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:55.481989 24755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:55.483172 24755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:55.495829 24760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:55.496178 24761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:55.497325 24755 server_base.cc:1034] running on GCE node
W20260621 07:58:55.497659 24763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:55.498176 24755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:55.498955 24755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:55.500125 24755 hybrid_clock.cc:648] HybridClock initialized: now 1782028735500053 us; error 86 us; skew 500 ppm
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:55.503847 24755 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:58:55.504976 24755 webserver.cc:460] Webserver started at http://127.11.223.131:44657/ using document root <none> and password file <none>
I20260621 07:58:55.505728 24755 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:55.505857 24755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:58:55.506137 24755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:55.522513 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "0adcb14c22e74c34914e0c8839c88a5c"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "2a7ddd1de8ca2bd9894bc84e8e0bbbed"
server_key_iv: "f62f4f7f3bec80d5efeb9bd712322576"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.523535 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "0adcb14c22e74c34914e0c8839c88a5c"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "2a7ddd1de8ca2bd9894bc84e8e0bbbed"
server_key_iv: "f62f4f7f3bec80d5efeb9bd712322576"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.529768 24755 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260621 07:58:55.548709 24770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.550580 24755 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260621 07:58:55.550710 24755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "0adcb14c22e74c34914e0c8839c88a5c"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "2a7ddd1de8ca2bd9894bc84e8e0bbbed"
server_key_iv: "f62f4f7f3bec80d5efeb9bd712322576"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.550840 24755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:55.588279 24755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:55.592232 24755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:55.592515 24755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:55.593258 24755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:55.594318 24755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:55.594419 24755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.594518 24755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:55.594581 24755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.608546 24755 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:45909
I20260621 07:58:55.609905 24755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:58:55.611296 24883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:45909 every 8 connection(s)
I20260621 07:58:55.617159 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 24755
I20260621 07:58:55.617295 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:58:55.617663 12158 external_mini_cluster.cc:1428] Setting key 0057f737c2e001f3a361e264a42191c7
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:55.628003 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:55.614760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:57119 (local address 127.11.223.190:38229)
0621 07:58:55.614868 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:55.614873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:55.614889 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:58:55.615742 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:55.615746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:55.615804 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:55.615864 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:55.617612 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.618389 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.620129 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.620496 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.620575 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:55.623614 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:55.623617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:55.623620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:55.623647 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:55.625481 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:55.626144 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:55.626147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:55.626149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:55.626238 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:55.626659 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:55.626663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:55.626665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:55.626891 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:55.626969 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:55.627700 (+ 731us) server_negotiation.cc:299] Negotiation successful
0621 07:58:55.627826 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:58:55.629951 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:55.613634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38229 (local address 127.11.223.131:57119)
0621 07:58:55.614228 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:55.614424 (+ 196us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:55.615459 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:55.616291 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:55.616300 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:55.616695 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:55.617411 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:55.617431 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.618609 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.618612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:55.619976 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:55.619987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:55.620775 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:55.620778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:55.620962 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:55.621634 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:55.621654 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:55.623434 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:55.625623 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:55.625629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:55.625646 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:55.625989 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:55.626354 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:55.626357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:55.626359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:55.626543 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:55.627086 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:55.627092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:55.627410 (+ 318us) client_negotiation.cc:769] Sending connection context
0621 07:58:55.628516 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0621 07:58:55.628861 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":142,"threads_started":1}
I20260621 07:58:55.631577 24884 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38229
I20260621 07:58:55.631830 24884 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:55.632781 24884 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:55.633909 24400 ts_manager.cc:194] Registered new tserver with Master: 0adcb14c22e74c34914e0c8839c88a5c (127.11.223.131:45909)
I20260621 07:58:55.634658 24400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:57119
I20260621 07:58:55.645785 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:58:55.646606 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 24481
I20260621 07:58:55.653875 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 24621
I20260621 07:58:55.660826 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 24755
I20260621 07:58:55.668527 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 24368
I20260621 07:58:55.675977 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:38229
--webserver_interface=127.11.223.190
--webserver_port=43847
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:38229
--ranger_config_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:55.788700 24887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:55.788962 24887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:55.789002 24887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:55.793092 24887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:58:55.793151 24887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:55.793169 24887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:55.793187 24887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:58:55.793244 24887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:58:55.798520 24887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:38229
--ranger_config_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:38229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=43847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:55.799544 24887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:55.800658 24887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:55.809005 24893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:55.808993 24895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:55.808996 24892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:55.809314 24887 server_base.cc:1034] running on GCE node
I20260621 07:58:55.809919 24887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:55.811102 24887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:55.812304 24887 hybrid_clock.cc:648] HybridClock initialized: now 1782028735812289 us; error 25 us; skew 500 ppm
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:55.814986 24887 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:58:55.815925 24887 webserver.cc:460] Webserver started at http://127.11.223.190:43847/ using document root <none> and password file <none>
I20260621 07:58:55.816526 24887 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:55.816630 24887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:58:55 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:55.862329 24887 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:58:55.865073 24903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:55.866677 24887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:58:55.866811 24887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "78186cb46992438a804a290656a937ad"
format_stamp: "Formatted at 2026-06-21 07:58:53 on dist-test-slave-0pn3"
server_key: "d0e65c96be33bf9519881d057b69e7bf"
server_key_iv: "451b166f7fb991064b0168cf44084488"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:55.867542 24887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:55.892453 24887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:55.895558 24887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:55.895720 24887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:55.903319 24887 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:38229
I20260621 07:58:55.903324 24955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:38229 every 8 connection(s)
I20260621 07:58:55.904443 24887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:58:55.908675 24956 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:58:55.910286 24956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Bootstrap starting.
I20260621 07:58:55.914544 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 24887
I20260621 07:58:55.915007 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:39573
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:38229
--webserver_port=44915
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:58:55.926571 24956 log.cc:824] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:55.933422 24956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:58:55.933807 24956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Bootstrap complete.
I20260621 07:58:55.937907 24956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:55.938318 24956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78186cb46992438a804a290656a937ad, State: Initialized, Role: FOLLOWER
I20260621 07:58:55.938827 24956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:55.938969 24956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:58:55.939052 24956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:58:55.939162 24956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:58:55.941144 24956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:55.941476 24956 leader_election.cc:302] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78186cb46992438a804a290656a937ad; no voters:
I20260621 07:58:55.941789 24956 leader_election.cc:288] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:58:55.941871 24960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:58:55.942070 24960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [term 2 LEADER]: Becoming Leader. State: Replica: 78186cb46992438a804a290656a937ad, State: Running, Role: LEADER
I20260621 07:58:55.942469 24960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } }
I20260621 07:58:55.943035 24956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:58:55.944847 24961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78186cb46992438a804a290656a937ad. Latest consensus state: current_term: 2 leader_uuid: "78186cb46992438a804a290656a937ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } } }
I20260621 07:58:55.944965 24961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: This master's current role is: LEADER
I20260621 07:58:55.945154 24962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "78186cb46992438a804a290656a937ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78186cb46992438a804a290656a937ad" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38229 } } }
I20260621 07:58:55.945250 24962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad [sys.catalog]: This master's current role is: LEADER
I20260621 07:58:55.945439 24956 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260621 07:58:55.948873 24975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:58:55.952862 24975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:58:55.953694 24975 catalog_manager.cc:1259] Loaded cluster ID: a47dc5a19a2045179399ab333d93ea5a
I20260621 07:58:55.953822 24975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:58:55.956985 24975 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:58:55.957759 24975 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Loaded TSK: 0
I20260621 07:58:55.959249 24975 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 07:58:56.062487 24958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:56.062860 24958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:56.062935 24958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:56.069128 24958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:56.069239 24958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:56.069362 24958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:58:56.077575 24958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:39573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=44915
--tserver_master_addrs=127.11.223.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:56.079057 24958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:56.080474 24958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:56.096335 24994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:56.098361 24996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:56.098340 24993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:56.099191 24958 server_base.cc:1034] running on GCE node
I20260621 07:58:56.099692 24958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:56.100469 24958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:56.107900 24958 hybrid_clock.cc:648] HybridClock initialized: now 1782028736107865 us; error 48 us; skew 500 ppm
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:56.111791 24958 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:58:56.113144 24958 webserver.cc:460] Webserver started at http://127.11.223.129:44915/ using document root <none> and password file <none>
I20260621 07:58:56.113880 24958 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:56.113997 24958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:56.141347 24958 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:58:56.144558 25004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.146196 24958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:58:56.146314 24958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "e42038c9e1bf4700919763b5d4c1d9b7"
format_stamp: "Formatted at 2026-06-21 07:58:54 on dist-test-slave-0pn3"
server_key: "f1e4e4b3c189a142d016b85192d35427"
server_key_iv: "b0a4abb16e21e1a2609adfeb185823f4"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:56.147075 24958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:56.171005 24958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:56.174350 24958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:56.174607 24958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:56.175333 24958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:56.176419 24958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:56.176508 24958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.176594 24958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:56.176646 24958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.188004 24958 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:39573
I20260621 07:58:56.189513 24958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:58:56.191001 25117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:39573 every 8 connection(s)
I20260621 07:58:56.194774 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 24958
I20260621 07:58:56.195272 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:43439
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:38229
--webserver_port=40643
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:56.216122 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.193852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52989 (local address 127.11.223.190:38229)
0621 07:58:56.194357 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:56.194375 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:56.194407 (+ 32us) server_negotiation.cc:407] Connection header received
0621 07:58:56.194647 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:56.194657 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:56.195009 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:56.195354 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:56.197053 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.198623 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.200370 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.201085 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.201228 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.206849 (+ 5621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:56.206856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:56.206875 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:56.206910 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:56.209260 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.212787 (+ 3527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.212813 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.212820 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.212986 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.213416 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.213419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.213421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.213918 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:56.214032 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:56.214758 (+ 726us) server_negotiation.cc:299] Negotiation successful
0621 07:58:56.215076 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":126,"threads_started":1}
I20260621 07:58:56.217993 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.192302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38229 (local address 127.11.223.129:52989)
0621 07:58:56.192852 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:56.192917 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:56.193989 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:56.195861 (+ 1872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:56.195870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:56.196222 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:56.196859 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:56.196880 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.198786 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.198790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:56.200177 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:56.200189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.201489 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.201493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:56.201700 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.202394 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:56.202414 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:56.206584 (+ 4170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:56.209417 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:56.209424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:56.209443 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:56.209890 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:56.213122 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:56.213125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:56.213128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:56.213312 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:56.214101 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:56.214107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:56.214406 (+ 299us) client_negotiation.cc:769] Sending connection context
0621 07:58:56.216654 (+ 2248us) client_negotiation.cc:240] Negotiation successful
0621 07:58:56.216988 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
I20260621 07:58:56.220782 25118 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38229
I20260621 07:58:56.221025 25118 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.221592 25118 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
W20260621 07:58:56.223843 25118 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:38229 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:58:56.223963 25118 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.224123 25118 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:56.225001 25118 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.225153 25118 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
W20260621 07:58:56.225883 25118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:58:56.338721 25122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:56.339182 25122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:56.339319 25122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:56.345166 25122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:56.345248 25122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:56.345347 25122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:58:56.352831 25122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:43439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=40643
--tserver_master_addrs=127.11.223.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:56.354360 25122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:56.355683 25122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:56.366714 25130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:56.367674 25128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:56.369642 25122 server_base.cc:1034] running on GCE node
W20260621 07:58:56.372738 25127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:56.373320 25122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:56.374092 25122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:56.389590 25122 hybrid_clock.cc:648] HybridClock initialized: now 1782028736389537 us; error 59 us; skew 500 ppm
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:56.393146 25122 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:58:56.394249 25122 webserver.cc:460] Webserver started at http://127.11.223.130:40643/ using document root <none> and password file <none>
I20260621 07:58:56.394837 25122 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:56.394938 25122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:56.416074 25122 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260621 07:58:56.420131 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.422942 25122 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260621 07:58:56.423158 25122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "6208150410dd4bd997a72c17c68ae53b"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "8f68bfdc20871140eb7bdbeb3b29d062"
server_key_iv: "9c48cd155416720a0ad07ed5377508a9"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:56.423878 25122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:56.462775 25122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:56.465934 25122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:56.466118 25122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:56.466773 25122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:56.467737 25122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:56.467792 25122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.467842 25122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:56.467871 25122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.478366 25122 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:43439
I20260621 07:58:56.478423 25252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:43439 every 8 connection(s)
I20260621 07:58:56.479583 25122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:58:56.487195 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25122
I20260621 07:58:56.487677 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.223.148:56009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:45909
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:38229
--webserver_port=44657
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:56.496634 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.483673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:51415 (local address 127.11.223.190:38229)
0621 07:58:56.483799 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:56.483803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:56.483820 (+ 17us) server_negotiation.cc:407] Connection header received
0621 07:58:56.484511 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:56.484515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:56.484573 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:56.484644 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:56.486158 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.486918 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.488750 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.489128 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.489219 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.492162 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:56.492166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:56.492169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:56.492195 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:56.494083 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.494743 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.494747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.494749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.494839 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.495268 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.495271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.495273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.495496 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:56.495595 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:56.496292 (+ 697us) server_negotiation.cc:299] Negotiation successful
0621 07:58:56.496467 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:58:56.498504 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.482529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38229 (local address 127.11.223.130:51415)
0621 07:58:56.483088 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:56.483169 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:56.484215 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:56.485038 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:56.485047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:56.485391 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:56.485990 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:56.486007 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.487138 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.487142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:56.488595 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:56.488606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.489430 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.489434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:56.489631 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.490351 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:56.490371 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:56.491947 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:56.494210 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:56.494218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:56.494237 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:56.494622 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:56.494945 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:56.494951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:56.494956 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:56.495149 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:56.495653 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:56.495659 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:56.495971 (+ 312us) client_negotiation.cc:769] Sending connection context
0621 07:58:56.497101 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0621 07:58:56.497437 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":145,"threads_started":1}
I20260621 07:58:56.500190 25253 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38229
I20260621 07:58:56.500512 25253 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.501125 25253 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
W20260621 07:58:56.502681 25253 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:38229 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260621 07:58:56.502800 25253 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.502952 25253 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:56.503718 25253 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.503891 25253 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
W20260621 07:58:56.504652 25253 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:58:56.620533 25256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:56.620867 25256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:56.620934 25256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:56.625846 25256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:56.625929 25256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:56.626032 25256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:58:56.632287 25256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.223.148:56009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:45909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=44657
--tserver_master_addrs=127.11.223.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:58:56.633730 25256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:58:56.634945 25256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:58:56.645900 25261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:56.646428 25256 server_base.cc:1034] running on GCE node
W20260621 07:58:56.648069 25264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:58:56.651930 25262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:58:56.652530 25256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:58:56.653426 25256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:58:56.654651 25256 hybrid_clock.cc:648] HybridClock initialized: now 1782028736654626 us; error 40 us; skew 500 ppm
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:56.658360 25256 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:58:56.659381 25256 webserver.cc:460] Webserver started at http://127.11.223.131:44657/ using document root <none> and password file <none>
I20260621 07:58:56.660146 25256 fs_manager.cc:359] Metadata directory not provided
I20260621 07:58:56.660249 25256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:56.680091 25256 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260621 07:58:56.683243 25272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.684847 25256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:58:56.684980 25256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "0adcb14c22e74c34914e0c8839c88a5c"
format_stamp: "Formatted at 2026-06-21 07:58:55 on dist-test-slave-0pn3"
server_key: "2a7ddd1de8ca2bd9894bc84e8e0bbbed"
server_key_iv: "f62f4f7f3bec80d5efeb9bd712322576"
server_key_version: "kuduclusterkey@0"
I20260621 07:58:56.685712 25256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:58:56.703970 25256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:58:56.707221 25256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:58:56.707465 25256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:58:56.708205 25256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:58:56.709352 25256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:58:56.709448 25256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.709539 25256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:58:56.709599 25256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:58:56.720862 25256 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:45909
I20260621 07:58:56.721282 25392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:45909 every 8 connection(s)
I20260621 07:58:56.722198 25256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:58:56.727183 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25256
Jun 21 07:58:56 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:58:56.737287 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.727831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.11.223.190:38229)
0621 07:58:56.728195 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:56.728205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:56.728221 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:58:56.728507 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:56.728513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:56.728564 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:56.728656 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:56.729720 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.730529 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.731742 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.732124 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.732230 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.734044 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:56.734046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:56.734048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:56.734084 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:56.735560 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.736013 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.736016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.736018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.736120 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.736454 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.736457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.736459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.736617 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:56.736828 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:56.736972 (+ 144us) server_negotiation.cc:299] Negotiation successful
0621 07:58:56.737090 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":115,"threads_started":1}
I20260621 07:58:56.738268 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.724487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38229 (local address 127.11.223.131:59037)
0621 07:58:56.724968 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:56.725020 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:56.725849 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:56.726419 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:56.726425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:56.726689 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:58:56.727368 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:56.727389 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.728611 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.728616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:56.729605 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:56.729614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.730326 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.730329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:56.730511 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.731165 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:58:56.731181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:58:56.732701 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:58:56.734956 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:56.734962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:56.734977 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:56.735338 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:56.735658 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:58:56.735661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:58:56.735663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:58:56.735837 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:58:56.736278 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:58:56.736284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:58:56.736626 (+ 342us) client_negotiation.cc:769] Sending connection context
0621 07:58:56.736854 (+ 228us) client_negotiation.cc:240] Negotiation successful
0621 07:58:56.737197 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":138,"threads_started":1}
I20260621 07:58:56.738943 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:56.724065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:59037 (local address 127.11.223.190:38229)
0621 07:58:56.724236 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:56.724242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:56.725156 (+ 914us) server_negotiation.cc:407] Connection header received
0621 07:58:56.726059 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:56.726063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:56.726129 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:56.726236 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:56.727533 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.728397 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.729756 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:56.730079 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:56.730172 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:56.732899 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:56.732903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:56.732906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:56.732932 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:56.734828 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.735464 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.735468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.735470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.735557 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:56.735962 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:56.735967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:56.735972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:56.736158 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:56.736279 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:56.738681 (+ 2402us) server_negotiation.cc:299] Negotiation successful
0621 07:58:56.738798 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 07:58:56.739867 25393 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38229
I20260621 07:58:56.740129 25393 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:56.740662 25393 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:56.742038 24920 ts_manager.cc:194] Registered new tserver with Master: 0adcb14c22e74c34914e0c8839c88a5c (127.11.223.131:45909)
I20260621 07:58:56.743270 24920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:59037
Jun 21 07:58:57 dist-test-slave-0pn3 krb5kdc[23924](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1782028737, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:58:57.226099 25118 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:57.226413 25118 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:57.227286 24920 ts_manager.cc:194] Registered new tserver with Master: e42038c9e1bf4700919763b5d4c1d9b7 (127.11.223.129:39573)
I20260621 07:58:57.228199 24920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:52989
Jun 21 07:58:57 dist-test-slave-0pn3 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028737, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for HTTP/127.11.223.148@KRBTEST.COM
I20260621 07:58:57.504899 25253 heartbeater.cc:464] Registering TS with master...
I20260621 07:58:57.505227 25253 heartbeater.cc:511] Master 127.11.223.190:38229 requested a full tablet report, sending...
I20260621 07:58:57.505977 24920 ts_manager.cc:194] Registered new tserver with Master: 6208150410dd4bd997a72c17c68ae53b (127.11.223.130:43439)
I20260621 07:58:57.506683 24920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:51415
I20260621 07:58:57.510049 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:58:57.526300 25407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.512706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52534 (local address 127.11.223.190:38229)
0621 07:58:57.516542 (+ 3836us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.516547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.516562 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:58:57.516606 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.516609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.516659 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.516746 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:58:57.517694 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.518536 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.519736 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.520083 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.520194 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.521965 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:58:57.521968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:58:57.521971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:58:57.521997 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:58:57.523763 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:57.524207 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:57.524211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:57.524212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:57.524291 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:58:57.525056 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:58:57.525059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:58:57.525060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:58:57.525264 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:58:57.525374 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.525996 (+ 622us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.526116 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3754,"thread_start_us":94,"threads_started":1}
I20260621 07:58:57.531203 24920 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:58:57.535934 24920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:58:57.562559 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.553730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.11.223.130:43439)
0621 07:58:57.555229 (+ 1499us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.555236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.555536 (+ 300us) server_negotiation.cc:407] Connection header received
0621 07:58:57.555617 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.555622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.555825 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.555958 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:57.556461 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.557510 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.559498 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.560288 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.560420 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.560546 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.562313 (+ 1767us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.562415 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1381,"thread_start_us":100,"threads_started":1}
I20260621 07:58:57.562906 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.553571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43439 (local address 127.0.0.1:34148)
0621 07:58:57.555356 (+ 1785us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.555369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.555451 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.556130 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:57.556146 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:57.556169 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:57.556301 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.556309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.557880 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.557883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.559392 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.559399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.560606 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.560610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.560724 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.560759 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 07:58:57.560858 (+ 99us) client_negotiation.cc:240] Negotiation successful
0621 07:58:57.560958 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1728,"thread_start_us":94,"threads_started":1}
I20260621 07:58:57.565570 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.552276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45909 (local address 127.0.0.1:45692)
0621 07:58:57.554779 (+ 2503us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.554830 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.554948 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.559148 (+ 4200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:57.559151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:57.559166 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:57.559248 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.559253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.561115 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.561119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.562170 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.562178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.565188 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.565191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.565297 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.565321 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:58:57.565370 (+ 49us) client_negotiation.cc:240] Negotiation successful
0621 07:58:57.565445 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2330,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260621 07:58:57.566466 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.553132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.11.223.131:45909)
0621 07:58:57.558701 (+ 5569us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.558708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.558731 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:58:57.558802 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.558806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.559014 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.559169 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:57.559987 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.561018 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.564218 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.565048 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.566013 (+ 965us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.566153 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.566205 (+ 52us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.566300 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5452,"thread_start_us":87,"threads_started":1}
I20260621 07:58:57.566509 25187 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba599f57cc74896bd0985ee1a740685 (DEFAULT_TABLE table=test-table [id=a2924def7a824e80bb9663ff5e16a9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:58:57.567021 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.554182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39573 (local address 127.0.0.1:50434)
0621 07:58:57.557704 (+ 3522us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.557716 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.557782 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.561889 (+ 4107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:57.561892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:57.561907 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:57.561977 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.561981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.564209 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.564213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.565334 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.565343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.566595 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.566599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.566713 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.566742 (+ 29us) client_negotiation.cc:769] Sending connection context
0621 07:58:57.566815 (+ 73us) client_negotiation.cc:240] Negotiation successful
0621 07:58:57.566881 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3470,"mutex_wait_us":42,"thread_start_us":79,"threads_started":1}
I20260621 07:58:57.567725 25187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba599f57cc74896bd0985ee1a740685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:57.568344 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.554321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.11.223.129:39573)
0621 07:58:57.561285 (+ 6964us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.561291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.561313 (+ 22us) server_negotiation.cc:407] Connection header received
0621 07:58:57.561381 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.561385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.561583 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.561747 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:57.562088 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.564077 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.565448 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.566278 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.566411 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.566528 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.567176 (+ 648us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.567268 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6855,"thread_start_us":77,"threads_started":1}
I20260621 07:58:57.569586 25327 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba599f57cc74896bd0985ee1a740685 (DEFAULT_TABLE table=test-table [id=a2924def7a824e80bb9663ff5e16a9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:58:57.570771 25327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba599f57cc74896bd0985ee1a740685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:57.570937 25038 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba599f57cc74896bd0985ee1a740685 (DEFAULT_TABLE table=test-table [id=a2924def7a824e80bb9663ff5e16a9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:58:57.571916 25038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba599f57cc74896bd0985ee1a740685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:58:57.582376 25420 tablet_bootstrap.cc:492] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Bootstrap starting.
I20260621 07:58:57.584770 25420 tablet_bootstrap.cc:654] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:57.585668 25420 log.cc:824] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:57.586485 25421 tablet_bootstrap.cc:492] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7: Bootstrap starting.
I20260621 07:58:57.588806 25421 tablet_bootstrap.cc:654] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:57.589031 25420 tablet_bootstrap.cc:492] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: No bootstrap required, opened a new log
I20260621 07:58:57.589226 25420 ts_tablet_manager.cc:1397] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260621 07:58:57.589658 25421 log.cc:824] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:57.590571 25423 tablet_bootstrap.cc:492] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: Bootstrap starting.
I20260621 07:58:57.592772 25423 tablet_bootstrap.cc:654] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: Neither blocks nor log segments found. Creating new log.
I20260621 07:58:57.593596 25423 log.cc:824] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:58:57.594748 25420 raft_consensus.cc:348] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.595108 25420 raft_consensus.cc:374] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:57.595219 25420 raft_consensus.cc:729] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0adcb14c22e74c34914e0c8839c88a5c, State: Initialized, Role: FOLLOWER
I20260621 07:58:57.595752 25420 consensus_queue.cc:260] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.596076 25421 tablet_bootstrap.cc:492] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7: No bootstrap required, opened a new log
I20260621 07:58:57.596282 25421 ts_tablet_manager.cc:1397] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260621 07:58:57.597005 25420 ts_tablet_manager.cc:1428] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260621 07:58:57.597390 25393 heartbeater.cc:503] Master 127.11.223.190:38229 was elected leader, sending a full tablet report...
I20260621 07:58:57.600142 25421 raft_consensus.cc:348] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.600477 25421 raft_consensus.cc:374] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:57.600575 25421 raft_consensus.cc:729] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e42038c9e1bf4700919763b5d4c1d9b7, State: Initialized, Role: FOLLOWER
I20260621 07:58:57.601101 25421 consensus_queue.cc:260] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.601222 25423 tablet_bootstrap.cc:492] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: No bootstrap required, opened a new log
I20260621 07:58:57.601402 25423 ts_tablet_manager.cc:1397] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260621 07:58:57.602187 25421 ts_tablet_manager.cc:1428] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260621 07:58:57.602576 25118 heartbeater.cc:503] Master 127.11.223.190:38229 was elected leader, sending a full tablet report...
I20260621 07:58:57.605336 25423 raft_consensus.cc:348] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.605613 25423 raft_consensus.cc:374] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:58:57.605713 25423 raft_consensus.cc:729] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6208150410dd4bd997a72c17c68ae53b, State: Initialized, Role: FOLLOWER
I20260621 07:58:57.606195 25423 consensus_queue.cc:260] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.606858 25253 heartbeater.cc:503] Master 127.11.223.190:38229 was elected leader, sending a full tablet report...
I20260621 07:58:57.607249 25423 ts_tablet_manager.cc:1428] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260621 07:58:57.614462 25427 raft_consensus.cc:482] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:58:57.614642 25427 raft_consensus.cc:504] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.615860 25427 leader_election.cc:288] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0adcb14c22e74c34914e0c8839c88a5c (127.11.223.131:45909), 6208150410dd4bd997a72c17c68ae53b (127.11.223.130:43439)
I20260621 07:58:57.621423 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.616555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:42221 (local address 127.11.223.130:43439)
0621 07:58:57.616656 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.616662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.617151 (+ 489us) server_negotiation.cc:407] Connection header received
0621 07:58:57.617310 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.617313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.617355 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.617397 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:57.617778 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.618529 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.620057 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.620696 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.620783 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.620830 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.621221 (+ 391us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.621299 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260621 07:58:57.621431 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.616417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:53979 (local address 127.11.223.131:45909)
0621 07:58:57.616539 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.616544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.616861 (+ 317us) server_negotiation.cc:407] Connection header received
0621 07:58:57.616962 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.616965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.617020 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.617102 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:58:57.617574 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.618386 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.620010 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.620681 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.620778 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.620832 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.621236 (+ 404us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.621310 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 07:58:57.621798 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.616054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45909 (local address 127.11.223.129:53979)
0621 07:58:57.616749 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.616763 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.616872 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.617256 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:57.617259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:57.617280 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:57.617427 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.617436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.618551 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.618555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.619862 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.619873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.620931 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.620934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.621059 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.621098 (+ 39us) client_negotiation.cc:769] Sending connection context
0621 07:58:57.621552 (+ 454us) client_negotiation.cc:240] Negotiation successful
0621 07:58:57.621625 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":11,"thread_start_us":97,"threads_started":1}
I20260621 07:58:57.622394 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.616169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43439 (local address 127.11.223.129:42221)
0621 07:58:57.617074 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.617086 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.617209 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.617513 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:58:57.617515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:58:57.617529 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:58:57.617657 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.617662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.618675 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.618678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.619920 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:58:57.619929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.620921 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.620924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:58:57.621072 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.621103 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 07:58:57.622175 (+ 1072us) client_negotiation.cc:240] Negotiation successful
0621 07:58:57.622246 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":38,"thread_start_us":141,"threads_started":1}
I20260621 07:58:57.622591 25347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba599f57cc74896bd0985ee1a740685" candidate_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0adcb14c22e74c34914e0c8839c88a5c" is_pre_election: true
I20260621 07:58:57.622905 25347 raft_consensus.cc:2454] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e42038c9e1bf4700919763b5d4c1d9b7 in term 0.
I20260621 07:58:57.623138 25199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba599f57cc74896bd0985ee1a740685" candidate_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6208150410dd4bd997a72c17c68ae53b" is_pre_election: true
I20260621 07:58:57.623422 25199 raft_consensus.cc:2454] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e42038c9e1bf4700919763b5d4c1d9b7 in term 0.
I20260621 07:58:57.623422 25006 leader_election.cc:302] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0adcb14c22e74c34914e0c8839c88a5c, e42038c9e1bf4700919763b5d4c1d9b7; no voters:
I20260621 07:58:57.623709 25427 raft_consensus.cc:2781] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:58:57.623802 25427 raft_consensus.cc:482] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:58:57.623852 25427 raft_consensus.cc:3037] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:57.625226 25427 raft_consensus.cc:504] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.625607 25427 leader_election.cc:288] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [CANDIDATE]: Term 1 election: Requested vote from peers 0adcb14c22e74c34914e0c8839c88a5c (127.11.223.131:45909), 6208150410dd4bd997a72c17c68ae53b (127.11.223.130:43439)
I20260621 07:58:57.625989 25347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba599f57cc74896bd0985ee1a740685" candidate_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0adcb14c22e74c34914e0c8839c88a5c"
I20260621 07:58:57.626029 25199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba599f57cc74896bd0985ee1a740685" candidate_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6208150410dd4bd997a72c17c68ae53b"
I20260621 07:58:57.626139 25199 raft_consensus.cc:3037] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:57.626124 25347 raft_consensus.cc:3037] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:58:57.627421 25347 raft_consensus.cc:2454] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e42038c9e1bf4700919763b5d4c1d9b7 in term 1.
I20260621 07:58:57.627421 25199 raft_consensus.cc:2454] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e42038c9e1bf4700919763b5d4c1d9b7 in term 1.
I20260621 07:58:57.627702 25006 leader_election.cc:302] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0adcb14c22e74c34914e0c8839c88a5c, e42038c9e1bf4700919763b5d4c1d9b7; no voters:
I20260621 07:58:57.627967 25427 raft_consensus.cc:2781] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:58:57.628227 25427 raft_consensus.cc:686] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [term 1 LEADER]: Becoming Leader. State: Replica: e42038c9e1bf4700919763b5d4c1d9b7, State: Running, Role: LEADER
I20260621 07:58:57.628610 25427 consensus_queue.cc:237] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } }
I20260621 07:58:57.632117 24920 catalog_manager.cc:5495] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e42038c9e1bf4700919763b5d4c1d9b7 (127.11.223.129). New cstate: current_term: 1 leader_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e42038c9e1bf4700919763b5d4c1d9b7" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39573 } health_report { overall_health: HEALTHY } } }
I20260621 07:58:57.659983 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.654827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.11.223.129:39573)
0621 07:58:57.654953 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0621 07:58:57.654958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:58:57.654973 (+ 15us) server_negotiation.cc:407] Connection header received
0621 07:58:57.655057 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:58:57.655061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:58:57.655126 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:58:57.655243 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:58:57.655514 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.656351 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.657743 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:58:57.658083 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:58:57.658188 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:58:57.658618 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:58:57.659216 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:58:57.659339 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0621 07:58:57.659760 (+ 421us) server_negotiation.cc:299] Negotiation successful
0621 07:58:57.659822 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 07:58:57.667289 25199 raft_consensus.cc:1261] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b [term 1 FOLLOWER]: Refusing update from remote peer e42038c9e1bf4700919763b5d4c1d9b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:58:57.668174 25431 consensus_queue.cc:1035] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208150410dd4bd997a72c17c68ae53b" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 43439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:58:57.671029 25347 raft_consensus.cc:1261] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 1 FOLLOWER]: Refusing update from remote peer e42038c9e1bf4700919763b5d4c1d9b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:58:57.671912 25427 consensus_queue.cc:1035] T 7ba599f57cc74896bd0985ee1a740685 P e42038c9e1bf4700919763b5d4c1d9b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0adcb14c22e74c34914e0c8839c88a5c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 45909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:58:57.679409 25434 mvcc.cc:205] Tried to move back new op lower bound from 7299189709477720064 to 7299189709329780736. Current Snapshot: MvccSnapshot[applied={T|T < 7299189709477720064}]
I20260621 07:58:57.688943 25435 mvcc.cc:205] Tried to move back new op lower bound from 7299189709477720064 to 7299189709329780736. Current Snapshot: MvccSnapshot[applied={T|T < 7299189709477720064}]
I20260621 07:58:57.707067 24920 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:58:57.707441 24920 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:58:57.712395 24920 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78186cb46992438a804a290656a937ad: Sending DeleteTablet for 3 replicas of tablet 7ba599f57cc74896bd0985ee1a740685
I20260621 07:58:57.714208 25327 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba599f57cc74896bd0985ee1a740685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:58:57 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:45692
I20260621 07:58:57.716540 25187 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba599f57cc74896bd0985ee1a740685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:58:57 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:34148
I20260621 07:58:57.719151 25038 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba599f57cc74896bd0985ee1a740685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:58:57 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50434
I20260621 07:58:57.720592 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 24958
I20260621 07:58:57.726691 25444 tablet_replica.cc:330] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: stopping tablet replica
I20260621 07:58:57.727586 25444 raft_consensus.cc:2229] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:58:57.728272 25444 raft_consensus.cc:2258] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:58:57.729203 25445 tablet_replica.cc:330] T 7ba599f57cc74896bd0985ee1a740685 P 6208150410dd4bd997a72c17c68ae53b: stopping tablet replica
I20260621 07:58:57.733249 25444 ts_tablet_manager.cc:1905] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:58:57.734562 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25122
W20260621 07:58:57.735323 24906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:39573 (error 108)
I20260621 07:58:57.745462 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:58:57.745080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39573 (local address 127.0.0.1:50458)
0621 07:58:57.745246 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.745351 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260621 07:58:57.745700 24906 catalog_manager.cc:4584] TS e42038c9e1bf4700919763b5d4c1d9b7 (127.11.223.129:39573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ba599f57cc74896bd0985ee1a740685: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39573: connect: Connection refused (error 111)
I20260621 07:58:57.748369 25444 ts_tablet_manager.cc:1918] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260621 07:58:57.751255 25418 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:58:57.750530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:43439 (local address 127.0.0.1:34150)
0621 07:58:57.750698 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.750721 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.750884 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.751039 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.130:43439: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":47}
I20260621 07:58:57.751227 25444 log.cc:1196] T 7ba599f57cc74896bd0985ee1a740685 P 0adcb14c22e74c34914e0c8839c88a5c: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/7ba599f57cc74896bd0985ee1a740685
I20260621 07:58:57.752058 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25256
W20260621 07:58:57.763561 25418 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:58:57.762310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:45909 (local address 127.0.0.1:45696)
0621 07:58:57.762694 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0621 07:58:57.762726 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0621 07:58:57.762936 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:58:57.763357 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.131:45909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":243}
I20260621 07:58:57.764917 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 24887
I20260621 07:58:57.776031 12158 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260621 07:58:58.120990 12158 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260621 07:58:58.121093 12158 mini_ranger.cc:66] Stopping Ranger...
I20260621 07:58:58.465725 12158 mini_ranger.cc:68] Stopped Ranger
2026-06-21 07:58:58.465 UTC [23954] LOG: received smart shutdown request
2026-06-21 07:58:58.469 UTC [23954] LOG: background worker "logical replication launcher" (PID 23963) exited with exit code 1
2026-06-21 07:58:58.469 UTC [23958] LOG: shutting down
2026-06-21 07:58:58.618 UTC [23954] LOG: database system is shut down
2026-06-21T07:58:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46373 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:58:58 dist-test-slave-0pn3 krb5kdc[25456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:58:58 dist-test-slave-0pn3 krb5kdc[25456](info): set up 2 sockets
Jun 21 07:58:58 dist-test-slave-0pn3 krb5kdc[25456](info): commencing operation
krb5kdc: starting...
W20260621 07:58:59.768603 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:58:59 dist-test-slave-0pn3 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:58:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:58:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:58:59.885514 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43937
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43937,127.11.223.189:43907,127.11.223.188:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:58:59.991477 25469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:58:59.991721 25469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:58:59.991760 25469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:58:59.995615 25469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:58:59.995674 25469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:58:59.995693 25469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:58:59.995709 25469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:58:59.995724 25469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:00.001020 25469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43937,127.11.223.189:43907,127.11.223.188:38343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:00.002148 25469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:00.003224 25469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:00.012066 25477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:00.012099 25475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:00.012295 25469 server_base.cc:1034] running on GCE node
W20260621 07:59:00.012066 25474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:00.012882 25469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:00.013897 25469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:00.015050 25469 hybrid_clock.cc:648] HybridClock initialized: now 1782028740015027 us; error 37 us; skew 500 ppm
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:00.017764 25469 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:00.018674 25469 webserver.cc:460] Webserver started at http://127.11.223.190:40535/ using document root <none> and password file <none>
I20260621 07:59:00.019243 25469 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:00.019330 25469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:00.019555 25469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:00.021564 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "46e826dc40884b7ea41ea77b25b610bc"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "fd09873b4cb164b699cbfb6e2953e60a"
server_key_iv: "f3ce7972181df62a2f1e39467bdfb917"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.022105 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "46e826dc40884b7ea41ea77b25b610bc"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "fd09873b4cb164b699cbfb6e2953e60a"
server_key_iv: "f3ce7972181df62a2f1e39467bdfb917"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.025624 25469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:59:00.027912 25484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:00.029102 25469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:00.029250 25469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "46e826dc40884b7ea41ea77b25b610bc"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "fd09873b4cb164b699cbfb6e2953e60a"
server_key_iv: "f3ce7972181df62a2f1e39467bdfb917"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.029357 25469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:00.048141 25469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:00.050913 25469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:00.051129 25469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:00.057907 25469 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:43937
I20260621 07:59:00.058228 25536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:43937 every 8 connection(s)
I20260621 07:59:00.058936 25469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:00.061416 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25469
I20260621 07:59:00.061579 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:59:00.061874 12158 external_mini_cluster.cc:1428] Setting key d723ad11669b4e9cb3e1d1440379cc20
I20260621 07:59:00.061987 25537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:00.065701 25537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.223.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.189@KRBTEST.COM" created.
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:00.080660 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.067827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43937 (local address 127.0.0.1:36580)
0621 07:59:00.068282 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.068355 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.069250 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.069924 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.069930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.070387 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.070629 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.070642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.071944 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.071951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.072791 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.072801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.073633 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.073637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.073755 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.074299 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.074322 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.075860 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.077697 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.077704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.077714 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.077920 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.078249 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.078252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.078254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.078361 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.078927 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.078932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.079068 (+ 136us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.079255 (+ 187us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.079609 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":100,"thread_start_us":137,"threads_started":1}
I20260621 07:59:00.080665 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.068705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.11.223.190:43937)
0621 07:59:00.069011 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.069023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.069073 (+ 50us) server_negotiation.cc:407] Connection header received
0621 07:59:00.069576 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.069580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.069747 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.069927 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.070780 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.071776 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.072911 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.073477 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.073586 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.076001 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.076005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.076013 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.076031 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.077562 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.078038 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.078044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.078046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.078148 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.078455 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.078458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.078459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.078817 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.078955 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.079310 (+ 355us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.079557 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260621 07:59:00.083336 25537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.084029 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.083781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40898)
0621 07:59:00.083887 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.083958 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260621 07:59:00.084193 25485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
I20260621 07:59:00.085062 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.084851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40912)
0621 07:59:00.084961 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.085004 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260621 07:59:00.085222 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:43907: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab.
Entry for principal kudu/127.11.223.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab.
I20260621 07:59:00.086663 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:43907
--webserver_interface=127.11.223.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab
--principal=kudu/127.11.223.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43937,127.11.223.189:43907,127.11.223.188:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:00.134487 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } attempt: 1
I20260621 07:59:00.135329 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.135008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40928)
0621 07:59:00.135195 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.135254 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 07:59:00.136112 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.135823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40934)
0621 07:59:00.135980 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.136020 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260621 07:59:00.136293 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:43907: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
I20260621 07:59:00.204447 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } attempt: 2
I20260621 07:59:00.205322 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.204998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40944)
0621 07:59:00.205158 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.205232 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260621 07:59:00.205170 25544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:00.205521 25544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:00.205626 25544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260621 07:59:00.206040 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.205832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40954)
0621 07:59:00.205927 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.205968 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260621 07:59:00.206187 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:43907: Network error: Client connection negotiation failed: client connection to 127.11.223.189:43907: connect: Connection refused (error 111)
W20260621 07:59:00.212375 25544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:00.212455 25544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:00.212543 25544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:00.212592 25544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:00.212637 25544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:00.222829 25544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43937,127.11.223.189:43907,127.11.223.188:38343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab
--principal=kudu/127.11.223.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:43907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.223.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:00.224388 25544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:00.225832 25544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:00.235956 25550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:00.236040 25552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:00.236076 25549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:00.236384 25544 server_base.cc:1034] running on GCE node
I20260621 07:59:00.236989 25544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:00.238157 25544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:00.239338 25544 hybrid_clock.cc:648] HybridClock initialized: now 1782028740239264 us; error 86 us; skew 500 ppm
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:00.242074 25544 init.cc:377] Logged in from keytab as kudu/127.11.223.189@KRBTEST.COM (short username kudu)
I20260621 07:59:00.242962 25544 webserver.cc:460] Webserver started at http://127.11.223.189:43543/ using document root <none> and password file <none>
I20260621 07:59:00.243505 25544 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:00.243593 25544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:00.243817 25544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:00.245823 25544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data/instance:
uuid: "245df8bf3d3049b994c97bc163115e41"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "c945af017fee3da50b12aca41f674bfd"
server_key_iv: "e56f95eddae49d269e1103583c47e481"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.246351 25544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal/instance:
uuid: "245df8bf3d3049b994c97bc163115e41"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "c945af017fee3da50b12aca41f674bfd"
server_key_iv: "e56f95eddae49d269e1103583c47e481"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.250013 25544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 07:59:00.252234 25559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:00.253412 25544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:00.253553 25544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal
uuid: "245df8bf3d3049b994c97bc163115e41"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "c945af017fee3da50b12aca41f674bfd"
server_key_iv: "e56f95eddae49d269e1103583c47e481"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.253656 25544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:00.274778 25544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:00.277506 25544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:00.277689 25544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:00.284487 25544 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.189:43907
I20260621 07:59:00.284785 25611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.189:43907 every 8 connection(s)
I20260621 07:59:00.285449 25544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
I20260621 07:59:00.288084 25612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:00.291662 25612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.293166 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25544
I20260621 07:59:00.293303 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-1/wal/instance
I20260621 07:59:00.293654 12158 external_mini_cluster.cc:1428] Setting key e36f852b55c4178f2138868e354d61d7
I20260621 07:59:00.297384 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } attempt: 3
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.188@KRBTEST.COM" created.
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 07:59:00.307502 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.293941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43937 (local address 127.0.0.1:36592)
0621 07:59:00.294374 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.294420 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.295186 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.295733 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.295740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.296143 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.296844 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.296865 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.297863 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.297866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.299076 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.299084 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.299603 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.299608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.299769 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.300409 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.300423 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.302363 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.304118 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.304124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.304137 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.304517 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.304850 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.304855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.304860 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.305053 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.305479 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.305487 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.305800 (+ 313us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.306009 (+ 209us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.306371 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":149,"threads_started":1}
I20260621 07:59:00.307773 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.293575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.11.223.190:43937)
0621 07:59:00.293743 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.293748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.294538 (+ 790us) server_negotiation.cc:407] Connection header received
0621 07:59:00.295385 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.295388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.295447 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.295555 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.297042 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.297716 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.299203 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.299452 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.299563 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.302550 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.302554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.302556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.302588 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.303985 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.304629 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.304634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.304638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.304738 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.305170 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.305173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.305175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.305373 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.305510 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.307417 (+ 1907us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.307561 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 07:59:00.309489 25612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.309731 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.297760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40962)
0621 07:59:00.297933 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.297946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.298031 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.298712 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.298717 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.298925 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.299067 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.299076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.300610 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.300614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.301353 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.301363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.302005 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.302009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.302127 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.302578 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.302602 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.305556 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.307753 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.307756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.307758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.308060 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.308612 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.308614 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.308616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.308711 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.309256 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.309259 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.309351 (+ 92us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.309456 (+ 105us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.309581 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"spinlock_wait_cycles":2944}
I20260621 07:59:00.309957 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.297886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.11.223.189:43907)
0621 07:59:00.298264 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.298274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.298297 (+ 23us) server_negotiation.cc:407] Connection header received
0621 07:59:00.298334 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.298339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.298521 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.298682 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.299221 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.300463 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.301473 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.301826 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.301984 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.305698 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.305705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.305717 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.305769 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.307630 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.308399 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.308408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.308413 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.308512 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.308813 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.308819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.308824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.309139 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.309286 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.309583 (+ 297us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.309758 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"spinlock_wait_cycles":4352,"thread_start_us":181,"threads_started":1}
I20260621 07:59:00.311776 25537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.312423 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.312200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38084)
0621 07:59:00.312297 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.312365 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260621 07:59:00.313112 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.312876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38092)
0621 07:59:00.313002 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.313044 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260621 07:59:00.313269 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:38343: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 07:59:00.320436 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.310321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40970)
0621 07:59:00.310573 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.310588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.310678 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.311100 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.311106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.311305 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.311444 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.311455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.312489 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.312494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.313349 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.313360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.313770 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.313774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.313884 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.314281 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.314292 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.316664 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.318847 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.318853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.318856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.319043 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.319402 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.319405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.319407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.319488 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.319887 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.319890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.319976 (+ 86us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.320098 (+ 122us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.320231 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260621 07:59:00.320478 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.310572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.11.223.189:43907)
0621 07:59:00.310881 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.310889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.310902 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:59:00.310932 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.310935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.310974 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.311060 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.311576 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.312379 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.313453 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.313662 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.313735 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.316788 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.316790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.316792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.316810 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.318739 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.319144 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.319149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.319151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.319267 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.319587 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.319590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.319592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.319771 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.319888 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.320142 (+ 254us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.320287 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":248,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab.
Entry for principal kudu/127.11.223.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab.
I20260621 07:59:00.321352 25612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.321604 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:38343
--webserver_interface=127.11.223.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab
--principal=kudu/127.11.223.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43937,127.11.223.189:43907,127.11.223.188:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:00.321993 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.321779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38108)
0621 07:59:00.321862 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.321934 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260621 07:59:00.322187 25560 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
I20260621 07:59:00.322986 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.322745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38118)
0621 07:59:00.322882 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.322932 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260621 07:59:00.323146 25612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:38343: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
I20260621 07:59:00.344388 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } attempt: 1
I20260621 07:59:00.345219 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.344928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38128)
0621 07:59:00.345073 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.345142 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 07:59:00.345953 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.345739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38132)
0621 07:59:00.345843 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.345888 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260621 07:59:00.346125 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:38343: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
I20260621 07:59:00.372396 25612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } attempt: 1
I20260621 07:59:00.373626 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.373188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38148)
0621 07:59:00.373442 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.373539 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260621 07:59:00.374436 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.374209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38158)
0621 07:59:00.374333 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.374378 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260621 07:59:00.374571 25612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:38343: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
I20260621 07:59:00.421305 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } attempt: 2
I20260621 07:59:00.422231 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.421846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38160)
0621 07:59:00.422031 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.422116 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260621 07:59:00.422998 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.422764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38166)
0621 07:59:00.422877 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.422926 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260621 07:59:00.423143 25537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:38343: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
W20260621 07:59:00.431473 25620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:00.431757 25620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:00.431828 25620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:00.436090 25620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:00.436142 25620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:00.436173 25620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:00.436226 25620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:00.436254 25620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:00.441759 25620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43937,127.11.223.189:43907,127.11.223.188:38343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab
--principal=kudu/127.11.223.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:38343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.223.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:00.442767 25620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:00.442711 25612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } attempt: 2
I20260621 07:59:00.443557 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.443272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38180)
0621 07:59:00.443408 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.443465 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260621 07:59:00.444053 25620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:59:00.444281 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.444052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38192)
0621 07:59:00.444168 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.444230 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260621 07:59:00.444417 25612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:38343: Network error: Client connection negotiation failed: client connection to 127.11.223.188:38343: connect: Connection refused (error 111)
W20260621 07:59:00.452260 25625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:00.452278 25628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:00.452378 25620 server_base.cc:1034] running on GCE node
W20260621 07:59:00.452252 25626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:00.453090 25620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:00.454159 25620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:00.455322 25620 hybrid_clock.cc:648] HybridClock initialized: now 1782028740455289 us; error 44 us; skew 500 ppm
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:00.458204 25620 init.cc:377] Logged in from keytab as kudu/127.11.223.188@KRBTEST.COM (short username kudu)
I20260621 07:59:00.459019 25620 webserver.cc:460] Webserver started at http://127.11.223.188:36953/ using document root <none> and password file <none>
I20260621 07:59:00.459524 25620 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:00.459586 25620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:00.459743 25620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:00.461725 25620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data/instance:
uuid: "d8488bbadf2d4398bebed12b1745c197"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "bf50782a3bdfeca4437a62ec8f07837e"
server_key_iv: "d6135fe91f243eb957dcb04d5bea5473"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.462225 25620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal/instance:
uuid: "d8488bbadf2d4398bebed12b1745c197"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "bf50782a3bdfeca4437a62ec8f07837e"
server_key_iv: "d6135fe91f243eb957dcb04d5bea5473"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.465591 25620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:59:00.467774 25635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:00.469018 25620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:59:00.469118 25620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal
uuid: "d8488bbadf2d4398bebed12b1745c197"
format_stamp: "Formatted at 2026-06-21 07:59:00 on dist-test-slave-0pn3"
server_key: "bf50782a3bdfeca4437a62ec8f07837e"
server_key_iv: "d6135fe91f243eb957dcb04d5bea5473"
server_key_version: "encryptionkey@0"
I20260621 07:59:00.469210 25620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:00.479694 25620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:00.482892 25620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:00.483034 25620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:00.489833 25620 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.188:38343
I20260621 07:59:00.490147 25687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.188:38343 every 8 connection(s)
I20260621 07:59:00.490809 25620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
I20260621 07:59:00.493489 25688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:00.497255 25688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.497893 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25620
I20260621 07:59:00.498050 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1782028587408853-12158-0/minicluster-data/master-2/wal/instance
I20260621 07:59:00.498335 12158 external_mini_cluster.cc:1428] Setting key 957a520011f5c68e695048c6a52da954
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
I20260621 07:59:00.511445 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.499243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.11.223.190:43937)
0621 07:59:00.499424 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.499428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.500234 (+ 806us) server_negotiation.cc:407] Connection header received
0621 07:59:00.501008 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.501011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.501061 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.501149 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.502242 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.503046 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.504403 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.504729 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.504847 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.507546 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.507550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.507553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.507580 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.509224 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.509749 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.509751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.509752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.509824 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.510192 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.510198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.510201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.510460 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.510541 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.511119 (+ 578us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.511255 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:59:00.512636 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.499635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43937 (local address 127.0.0.1:36600)
0621 07:59:00.500081 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.500127 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.500809 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.501242 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.501248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.501730 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.502099 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.502114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.503150 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.503153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.504272 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.504283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.504881 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.504884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.505022 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.505685 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.505710 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.507399 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.509330 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.509338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.509348 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.509649 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.509912 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.509916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.509918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.510101 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.510562 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.510568 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.510875 (+ 307us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.511088 (+ 213us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.511400 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"spinlock_wait_cycles":2304,"thread_start_us":137,"threads_started":1}
I20260621 07:59:00.512636 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.499636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.11.223.188:38343)
0621 07:59:00.500031 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.500045 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.500075 (+ 30us) server_negotiation.cc:407] Connection header received
0621 07:59:00.500790 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.500807 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.501026 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.501218 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.502102 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.503314 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.504506 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.505029 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.505103 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.508036 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.508043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.508057 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.508083 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.510004 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.510481 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.510485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.510486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.510594 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.510837 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.510840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.510842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.511091 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.511265 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.511467 (+ 202us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.511615 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"spinlock_wait_cycles":25216,"thread_start_us":144,"threads_started":1}
I20260621 07:59:00.513911 25688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:00.522290 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.514466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:43907 (local address 127.0.0.1:40974)
0621 07:59:00.514559 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.514571 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.514640 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.514906 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.514909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.515081 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.515191 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.515198 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.516052 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.516055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.516688 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.516698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.517127 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.517130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.517225 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.517701 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.517712 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.518993 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.520796 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.520801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.520806 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.521070 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.521362 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.521367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.521369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.521464 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.521899 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.521901 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.521968 (+ 67us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.522042 (+ 74us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.522140 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260621 07:59:00.522392 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.514578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.11.223.189:43907)
0621 07:59:00.514711 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.514716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.514728 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:59:00.514770 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.514773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.514809 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.514894 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.515332 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.515940 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.516804 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.517024 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.517097 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.519121 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.519125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.519128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.519167 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.520673 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.521171 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.521174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.521175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.521268 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.521557 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.521561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.521566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.521794 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.521907 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.522115 (+ 208us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.522246 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260621 07:59:00.522341 25537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } attempt: 3
I20260621 07:59:00.523196 25688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } has no permanent_uuid. Determining permanent_uuid...
I20260621 07:59:00.523974 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.515169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.11.223.190:43937)
0621 07:59:00.515307 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.515310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.515322 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:59:00.515412 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.515414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.515452 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.515527 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.516051 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.516559 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.517434 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.517754 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.517828 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.520536 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.520541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.520547 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.520576 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.522159 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.522791 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.522794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.522795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.522877 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.523146 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.523149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.523150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.523428 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.523649 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.523751 (+ 102us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.523843 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 07:59:00.531493 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.522752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38204)
0621 07:59:00.522872 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.522885 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.522956 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.523248 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.523251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.523413 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.523501 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.523507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.524393 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.524398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.525140 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.525148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.525795 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.525801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.525884 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.526316 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.526330 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.527614 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.529895 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.529899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.529902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.530170 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.530437 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.530440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.530442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.530525 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.530950 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.530953 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.531062 (+ 109us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.531155 (+ 93us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.531294 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260621 07:59:00.532119 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.522951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.11.223.188:38343)
0621 07:59:00.523081 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.523086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.523097 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:59:00.523125 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.523127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.523166 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.523238 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.523609 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.524275 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.525258 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.525621 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.525766 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.528029 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.528037 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.528040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.528062 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.529780 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.530269 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.530271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.530272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.530335 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.530622 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.530625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.530627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.530833 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.531701 (+ 868us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.531772 (+ 71us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.531907 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46,"spinlock_wait_cycles":2304}
I20260621 07:59:00.533938 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.523609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38210)
0621 07:59:00.523763 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.523774 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.523834 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.524559 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.524561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.524724 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.524792 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.524797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.526119 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.526125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.526805 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.526815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.527494 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.527498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.527589 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.528055 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.528067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.529780 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.532291 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.532294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.532295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.532556 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.532882 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.532885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.532887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.532984 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.533450 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.533452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.533522 (+ 70us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.533630 (+ 108us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.533752 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260621 07:59:00.534018 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.523961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.11.223.188:38343)
0621 07:59:00.524275 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.524281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.524291 (+ 10us) server_negotiation.cc:407] Connection header received
0621 07:59:00.524354 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.524359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.524408 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.524482 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.524907 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.526010 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.526918 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.527298 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.527396 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.530328 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.530331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.530333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.530359 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.532166 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.532673 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.532676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.532678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.532792 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.533100 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.533103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.533104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.533335 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.533420 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.533709 (+ 289us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.533826 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":146,"threads_started":1}
I20260621 07:59:00.535578 25612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } attempt: 3
I20260621 07:59:00.535720 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.525423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.11.223.189:43907)
0621 07:59:00.525522 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.525525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.525597 (+ 72us) server_negotiation.cc:407] Connection header received
0621 07:59:00.525705 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.525708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.525748 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.525815 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.526564 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.527496 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.528302 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.528682 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.528803 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.531797 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.531801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.531803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.531824 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.533696 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.534535 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.534540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.534542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.534624 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.534889 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.534893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.534895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.535071 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.535291 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.535401 (+ 110us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.535510 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260621 07:59:00.535809 25537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: Bootstrap starting.
I20260621 07:59:00.538682 25688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: Bootstrap starting.
I20260621 07:59:00.538748 25537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:00.539211 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:59:00.539644 25537 log.cc:824] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:00.541149 25688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: Neither blocks nor log segments found. Creating new log.
Jun 21 07:59:00 dist-test-slave-0pn3 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028740, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
I20260621 07:59:00.541925 25688 log.cc:824] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:00.543151 25537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: No bootstrap required, opened a new log
I20260621 07:59:00.545539 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.536020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:38343 (local address 127.0.0.1:38218)
0621 07:59:00.536113 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:00.536128 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:00.536198 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:00.536586 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:00.536589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:00.536765 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:00.536967 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.536975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.538346 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.538349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.539131 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:00.539153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.539735 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.539740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:00.539836 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.540391 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:00.540407 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:00.541912 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:00.543890 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.543893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.543895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.544164 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.544475 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:00.544478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:00.544480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:00.544598 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:00.545052 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:00.545055 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:00.545146 (+ 91us) client_negotiation.cc:769] Sending connection context
0621 07:59:00.545244 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:59:00.545379 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260621 07:59:00.546123 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.536179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.11.223.188:38343)
0621 07:59:00.536381 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.536385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.536398 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:59:00.536426 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.536429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.536466 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.537069 (+ 603us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.537155 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.538210 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.539283 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.539604 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.540039 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.542050 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.542054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.542056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.542080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.543759 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.544275 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.544278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.544280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.544381 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.544716 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.544719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.544720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.544921 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.545008 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.545849 (+ 841us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.545961 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260621 07:59:00.547475 25537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.547710 25537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:00.547766 25537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e826dc40884b7ea41ea77b25b610bc, State: Initialized, Role: FOLLOWER
I20260621 07:59:00.548302 25537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.549455 25612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41: Bootstrap starting.
I20260621 07:59:00.549726 25537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:00.550999 25688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: No bootstrap required, opened a new log
W20260621 07:59:00.552603 25490 tablet.cc:2369] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:59:00.553257 25612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:00.554096 25612 log.cc:824] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41: Log is configured to *not* fsync() on all Append() calls
W20260621 07:59:00.554924 25714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:59:00.554981 25714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:59:00.555011 25688 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.555163 25703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.555250 25703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.555244 25688 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:00.555289 25688 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8488bbadf2d4398bebed12b1745c197, State: Initialized, Role: FOLLOWER
I20260621 07:59:00.555732 25688 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.556244 25717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.556407 25717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.554580 25469 master_runner.cc:186] Error getting master registration for 127.11.223.189:43907: OK, instance_id { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" instance_seqno: 1782028740282544 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:59:00.558425 25469 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:59:00.559373 25688 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:00.560312 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.543131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36606 (local address 127.11.223.190:43937)
0621 07:59:00.543303 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.543308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.543321 (+ 13us) server_negotiation.cc:407] Connection header received
0621 07:59:00.543408 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.543411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.543453 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.543521 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.544573 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.545375 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.547196 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.547508 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.547574 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.549660 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.549665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.549670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.549698 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.551271 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.558775 (+ 7504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.558779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.558781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.558875 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.559188 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.559192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.559194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.559419 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.559525 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.559967 (+ 442us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.560104 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 07:59:00.562423 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.542599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.11.223.189:43907)
0621 07:59:00.542714 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.542718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.543683 (+ 965us) server_negotiation.cc:407] Connection header received
0621 07:59:00.543721 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.543724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.543767 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.543834 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.544680 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.545398 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.547336 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.547647 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.547732 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.558091 (+ 10359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.558096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.558099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.558172 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.560100 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.560685 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.560689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.560690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.560772 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.561073 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.561076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.561077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.561283 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.561522 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.562127 (+ 605us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.562260 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 07:59:00.564270 25612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41: No bootstrap required, opened a new log
W20260621 07:59:00.567461 25728 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:59:00.567857 25728 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:59:00.568405 25612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.568616 25612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:00.568673 25612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245df8bf3d3049b994c97bc163115e41, State: Initialized, Role: FOLLOWER
I20260621 07:59:00.569111 25612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.565979 25620 master_runner.cc:186] Error getting master registration for 127.11.223.189:43907: OK, instance_id { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" instance_seqno: 1782028740282544 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 07:59:00.569309 25620 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 07:59:00.569556 25730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.569679 25730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.570269 25612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:00.575909 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:00.542885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38230 (local address 127.11.223.188:38343)
0621 07:59:00.543013 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:00.543017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:00.545800 (+ 2783us) server_negotiation.cc:407] Connection header received
0621 07:59:00.545835 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:00.545838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:00.545877 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:00.545939 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:00.548717 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.549508 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.557705 (+ 8197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:00.557999 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:00.558132 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:00.562386 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:00.562390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:00.562394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:00.562421 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:00.573597 (+ 11176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.574154 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.574158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.574160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.574248 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:00.574568 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:00.574573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:00.574576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:00.574813 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:00.574959 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:00.575605 (+ 646us) server_negotiation.cc:299] Negotiation successful
0621 07:59:00.575740 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:59:00.579175 25544 master_runner.cc:263] No leader master found from master 245df8bf3d3049b994c97bc163115e41
I20260621 07:59:00.579275 25544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260621 07:59:00.588040 25741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:59:00.588217 25741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:59:00.590564 25730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:00.590694 25730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.591626 25730 leader_election.cc:288] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46e826dc40884b7ea41ea77b25b610bc (127.11.223.190:43937), d8488bbadf2d4398bebed12b1745c197 (127.11.223.188:38343)
I20260621 07:59:00.592521 25511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "245df8bf3d3049b994c97bc163115e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46e826dc40884b7ea41ea77b25b610bc" is_pre_election: true
I20260621 07:59:00.592829 25511 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245df8bf3d3049b994c97bc163115e41 in term 0.
I20260621 07:59:00.593222 25662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "245df8bf3d3049b994c97bc163115e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8488bbadf2d4398bebed12b1745c197" is_pre_election: true
I20260621 07:59:00.593473 25662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245df8bf3d3049b994c97bc163115e41 in term 0.
I20260621 07:59:00.594861 25560 leader_election.cc:302] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245df8bf3d3049b994c97bc163115e41, d8488bbadf2d4398bebed12b1745c197; no voters:
I20260621 07:59:00.595855 25730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:59:00.595940 25730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:59:00.595983 25730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:00.597429 25730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.597733 25730 leader_election.cc:288] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [CANDIDATE]: Term 1 election: Requested vote from peers 46e826dc40884b7ea41ea77b25b610bc (127.11.223.190:43937), d8488bbadf2d4398bebed12b1745c197 (127.11.223.188:38343)
I20260621 07:59:00.598421 25662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "245df8bf3d3049b994c97bc163115e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8488bbadf2d4398bebed12b1745c197"
I20260621 07:59:00.598536 25662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:00.600073 25511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "245df8bf3d3049b994c97bc163115e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46e826dc40884b7ea41ea77b25b610bc"
I20260621 07:59:00.600201 25511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:00.601933 25511 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245df8bf3d3049b994c97bc163115e41 in term 1.
I20260621 07:59:00.602468 25560 leader_election.cc:302] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245df8bf3d3049b994c97bc163115e41, 46e826dc40884b7ea41ea77b25b610bc; no voters:
I20260621 07:59:00.603422 25662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245df8bf3d3049b994c97bc163115e41 in term 1.
I20260621 07:59:00.604808 25730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:00.605050 25730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [term 1 LEADER]: Becoming Leader. State: Replica: 245df8bf3d3049b994c97bc163115e41, State: Running, Role: LEADER
I20260621 07:59:00.605438 25730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } }
I20260621 07:59:00.607270 25730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245df8bf3d3049b994c97bc163115e41. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.607427 25730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:00.607801 25745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:00.608846 25745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:00.617491 25662 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [term 1 FOLLOWER]: Refusing update from remote peer 245df8bf3d3049b994c97bc163115e41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:00.618233 25743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:00.622155 25717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245df8bf3d3049b994c97bc163115e41. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.622263 25717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.625428 25511 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [term 1 FOLLOWER]: Refusing update from remote peer 245df8bf3d3049b994c97bc163115e41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:00.625998 25743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:00.629937 25703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245df8bf3d3049b994c97bc163115e41. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.630053 25703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.636133 25743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.636269 25743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:00.639149 25703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.639263 25703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.640355 25746 mvcc.cc:205] Tried to move back new op lower bound from 7299189721561878528 to 7299189721523093504. Current Snapshot: MvccSnapshot[applied={T|T < 7299189721561878528}]
I20260621 07:59:00.641348 25743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.641461 25743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:00.642524 25717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "245df8bf3d3049b994c97bc163115e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e826dc40884b7ea41ea77b25b610bc" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43937 } } peers { permanent_uuid: "245df8bf3d3049b994c97bc163115e41" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 43907 } } peers { permanent_uuid: "d8488bbadf2d4398bebed12b1745c197" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 38343 } } }
I20260621 07:59:00.642633 25717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 07:59:00.644109 25749 mvcc.cc:205] Tried to move back new op lower bound from 7299189721561878528 to 7299189721523093504. Current Snapshot: MvccSnapshot[applied={T|T < 7299189721561878528}]
I20260621 07:59:00.645028 25747 mvcc.cc:205] Tried to move back new op lower bound from 7299189721561878528 to 7299189721523093504. Current Snapshot: MvccSnapshot[applied={T|T < 7299189721561878528}]
I20260621 07:59:00.646860 25745 catalog_manager.cc:1347] Generated new cluster ID: dc98b3132ecb4319856338443cbd80ca
I20260621 07:59:00.646914 25745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:59:00.686748 25745 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:59:00.688144 25745 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:59:00.717476 25745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 245df8bf3d3049b994c97bc163115e41: Generated new TSK 0
I20260621 07:59:00.720666 25745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:59:01.556198 25714 catalog_manager.cc:1259] Loaded cluster ID: dc98b3132ecb4319856338443cbd80ca
I20260621 07:59:01.556285 25714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: loading cluster ID for follower catalog manager: success
I20260621 07:59:01.558197 25714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: acquiring CA information for follower catalog manager: success
I20260621 07:59:01.559454 25714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 46e826dc40884b7ea41ea77b25b610bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:59:01.569939 25728 catalog_manager.cc:1259] Loaded cluster ID: dc98b3132ecb4319856338443cbd80ca
I20260621 07:59:01.570019 25728 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: loading cluster ID for follower catalog manager: success
I20260621 07:59:01.571913 25728 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: acquiring CA information for follower catalog manager: success
I20260621 07:59:01.573203 25728 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8488bbadf2d4398bebed12b1745c197: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 07:59:01.797401 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25469
I20260621 07:59:01.803601 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25544
I20260621 07:59:01.809340 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25620
2026-06-21T07:59:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3069 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:59:01 dist-test-slave-0pn3 krb5kdc[25760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:59:01 dist-test-slave-0pn3 krb5kdc[25760](info): set up 2 sockets
Jun 21 07:59:01 dist-test-slave-0pn3 krb5kdc[25760](info): commencing operation
krb5kdc: starting...
W20260621 07:59:02.824884 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:59:02 dist-test-slave-0pn3 krb5kdc[25760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:59:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:59:02.944504 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33135
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:03.053205 25773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:03.053478 25773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:03.053560 25773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:03.057547 25773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:03.057607 25773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:03.057627 25773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:03.057643 25773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:03.057701 25773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:03.063007 25773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:03.064109 25773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:03.065138 25773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:03.073556 25779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:03.073544 25781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:03.073642 25778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:03.073704 25773 server_base.cc:1034] running on GCE node
I20260621 07:59:03.074385 25773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:03.075454 25773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:03.076604 25773 hybrid_clock.cc:648] HybridClock initialized: now 1782028743076593 us; error 23 us; skew 500 ppm
Jun 21 07:59:03 dist-test-slave-0pn3 krb5kdc[25760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:03.079190 25773 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:03.080044 25773 webserver.cc:460] Webserver started at http://127.11.223.190:35235/ using document root <none> and password file <none>
I20260621 07:59:03.080639 25773 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:03.080699 25773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:03.080915 25773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:03.082922 25773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "7f1bb4abbeb547d3b55041e4070b7365"
format_stamp: "Formatted at 2026-06-21 07:59:03 on dist-test-slave-0pn3"
server_key: "5f1a64bbf34d6ac1ef85b311aa4dac3c"
server_key_iv: "888346fe627fbe7523040f9d3201b5ee"
server_key_version: "encryptionkey@0"
I20260621 07:59:03.083575 25773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "7f1bb4abbeb547d3b55041e4070b7365"
format_stamp: "Formatted at 2026-06-21 07:59:03 on dist-test-slave-0pn3"
server_key: "5f1a64bbf34d6ac1ef85b311aa4dac3c"
server_key_iv: "888346fe627fbe7523040f9d3201b5ee"
server_key_version: "encryptionkey@0"
I20260621 07:59:03.087185 25773 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260621 07:59:03.089491 25788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:03.090649 25773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:59:03.090788 25773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "7f1bb4abbeb547d3b55041e4070b7365"
format_stamp: "Formatted at 2026-06-21 07:59:03 on dist-test-slave-0pn3"
server_key: "5f1a64bbf34d6ac1ef85b311aa4dac3c"
server_key_iv: "888346fe627fbe7523040f9d3201b5ee"
server_key_version: "encryptionkey@0"
I20260621 07:59:03.090911 25773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:03.109513 25773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:03.112411 25773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:03.112607 25773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:03.119589 25773 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:33135
I20260621 07:59:03.119622 25840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:33135 every 8 connection(s)
I20260621 07:59:03.120638 25773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:03.123349 25841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:03.128947 25841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365: Bootstrap starting.
I20260621 07:59:03.130759 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25773
I20260621 07:59:03.130895 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:59:03.131178 12158 external_mini_cluster.cc:1428] Setting key 75304e91d96740ebc5af993b80678616
I20260621 07:59:03.131698 25841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:03.132682 25841 log.cc:824] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:03.134967 25841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365: No bootstrap required, opened a new log
I20260621 07:59:03.139617 25841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f1bb4abbeb547d3b55041e4070b7365" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33135 } }
I20260621 07:59:03.139842 25841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:03.139912 25841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1bb4abbeb547d3b55041e4070b7365, State: Initialized, Role: FOLLOWER
I20260621 07:59:03.140460 25841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f1bb4abbeb547d3b55041e4070b7365" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33135 } }
I20260621 07:59:03.140619 25841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:59:03.140689 25841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:59:03.140790 25841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 0 FOLLOWER]: Advancing to term 1
Jun 21 07:59:03 dist-test-slave-0pn3 krb5kdc[25760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:03.142015 25841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f1bb4abbeb547d3b55041e4070b7365" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33135 } }
I20260621 07:59:03.142449 25841 leader_election.cc:302] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f1bb4abbeb547d3b55041e4070b7365; no voters:
I20260621 07:59:03.142843 25841 leader_election.cc:288] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:59:03.142954 25846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:03.143200 25846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [term 1 LEADER]: Becoming Leader. State: Replica: 7f1bb4abbeb547d3b55041e4070b7365, State: Running, Role: LEADER
I20260621 07:59:03.143561 25846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f1bb4abbeb547d3b55041e4070b7365" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33135 } }
I20260621 07:59:03.144317 25841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:03.145689 25848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f1bb4abbeb547d3b55041e4070b7365. Latest consensus state: current_term: 1 leader_uuid: "7f1bb4abbeb547d3b55041e4070b7365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f1bb4abbeb547d3b55041e4070b7365" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33135 } } }
I20260621 07:59:03.145689 25847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f1bb4abbeb547d3b55041e4070b7365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f1bb4abbeb547d3b55041e4070b7365" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33135 } } }
I20260621 07:59:03.145849 25848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:03.145849 25847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:03.147755 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:03.132256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.11.223.190:33135)
0621 07:59:03.132752 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:03.132773 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:03.132816 (+ 43us) server_negotiation.cc:407] Connection header received
0621 07:59:03.133776 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:03.133800 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:03.134211 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:03.134580 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:03.135285 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:03.136857 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:03.137721 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:03.138548 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:03.138720 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:03.141421 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:03.141427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:03.141445 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:03.141476 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:03.143526 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:03.144067 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:03.144073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:03.144078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:03.144229 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:03.144564 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:03.144567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:03.144569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:03.145124 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:03.146159 (+ 1035us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:03.146339 (+ 180us) server_negotiation.cc:299] Negotiation successful
0621 07:59:03.146711 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":149,"threads_started":1}
W20260621 07:59:03.149464 25861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 07:59:03.149556 25861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 07:59:03.149647 25857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:03.150372 25857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:03.151376 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 07:59:03.155900 25857 catalog_manager.cc:1347] Generated new cluster ID: 16f849196e764e32aeca402d42e685be
I20260621 07:59:03.155967 25857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:59:03.169926 25857 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:59:03.170929 25857 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:59:03.182960 25857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f1bb4abbeb547d3b55041e4070b7365: Generated new TSK 0
I20260621 07:59:03.183722 25857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 07:59:03.207366 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25773
2026-06-21T07:59:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1396 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:59:03 dist-test-slave-0pn3 krb5kdc[25868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:59:03 dist-test-slave-0pn3 krb5kdc[25868](info): set up 2 sockets
Jun 21 07:59:03 dist-test-slave-0pn3 krb5kdc[25868](info): commencing operation
krb5kdc: starting...
W20260621 07:59:04.232756 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:59:04 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:59:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:59:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:59:04.351683 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44709
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:04.459251 25881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:04.459573 25881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:04.459661 25881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:04.463572 25881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:04.463665 25881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:04.463704 25881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:04.463740 25881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:04.463779 25881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:04.469085 25881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:04.470147 25881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:04.471164 25881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:04.479450 25887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:04.479431 25886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:04.479712 25881 server_base.cc:1034] running on GCE node
W20260621 07:59:04.479429 25889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:04.480376 25881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:04.481472 25881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:04.482666 25881 hybrid_clock.cc:648] HybridClock initialized: now 1782028744482632 us; error 50 us; skew 500 ppm
Jun 21 07:59:04 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:04.485456 25881 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:04.486428 25881 webserver.cc:460] Webserver started at http://127.11.223.190:40659/ using document root <none> and password file <none>
I20260621 07:59:04.487006 25881 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:04.487094 25881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:04.487326 25881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:04.489393 25881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "ff2712f0acd6425ca0192b0a11d6d6c6"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "4bbd69095542d333d79e1f06f9db6ca7"
server_key_iv: "fe83e515e0577d7c2eee5f388c41a0ce"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.489961 25881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "ff2712f0acd6425ca0192b0a11d6d6c6"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "4bbd69095542d333d79e1f06f9db6ca7"
server_key_iv: "fe83e515e0577d7c2eee5f388c41a0ce"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.493564 25881 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:59:04.495975 25896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:04.497236 25881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:59:04.497378 25881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ff2712f0acd6425ca0192b0a11d6d6c6"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "4bbd69095542d333d79e1f06f9db6ca7"
server_key_iv: "fe83e515e0577d7c2eee5f388c41a0ce"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.497486 25881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:04.521265 25881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:04.524246 25881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:04.524475 25881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:04.531697 25881 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:44709
I20260621 07:59:04.531976 25948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:44709 every 8 connection(s)
I20260621 07:59:04.532826 25881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:04.535574 25949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:04.538293 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25881
I20260621 07:59:04.538439 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:59:04.538806 12158 external_mini_cluster.cc:1428] Setting key 619743237f68f919fdb4352cd3f1468d
I20260621 07:59:04.541874 25949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap starting.
I20260621 07:59:04.544139 25949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:04.544955 25949 log.cc:824] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:04.547034 25949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: No bootstrap required, opened a new log
Jun 21 07:59:04 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:04.550868 25949 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:04.551170 25949 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:04.551239 25949 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2712f0acd6425ca0192b0a11d6d6c6, State: Initialized, Role: FOLLOWER
I20260621 07:59:04.551805 25949 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:04.551971 25949 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:59:04.552062 25949 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:59:04.552187 25949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:04.553395 25949 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:04.553767 25949 leader_election.cc:302] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff2712f0acd6425ca0192b0a11d6d6c6; no voters:
I20260621 07:59:04.554072 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:04.540356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.11.223.190:44709)
0621 07:59:04.540814 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:04.540828 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:04.540862 (+ 34us) server_negotiation.cc:407] Connection header received
0621 07:59:04.541456 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:04.541472 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:04.541797 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:04.542185 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:04.543044 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:04.544030 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:04.544991 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:04.545677 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:04.545835 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:04.548686 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:04.548692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:04.548704 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:04.548731 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:04.550936 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:04.551440 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:04.551444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:04.551449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:04.551561 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:04.551837 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:04.551839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:04.551840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:04.552386 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:04.552550 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:04.552839 (+ 289us) server_negotiation.cc:299] Negotiation successful
0621 07:59:04.553138 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":149,"threads_started":1}
I20260621 07:59:04.554102 25949 leader_election.cc:288] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:59:04.554180 25954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:04.554396 25954 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 LEADER]: Becoming Leader. State: Replica: ff2712f0acd6425ca0192b0a11d6d6c6, State: Running, Role: LEADER
I20260621 07:59:04.554701 25954 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:04.555328 25949 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:04.555881 25954 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } } }
I20260621 07:59:04.555992 25954 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:04.556174 25954 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff2712f0acd6425ca0192b0a11d6d6c6. Latest consensus state: current_term: 1 leader_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } } }
I20260621 07:59:04.556340 25954 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:04.556664 25959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:04.559767 25959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:04.565407 25959 catalog_manager.cc:1347] Generated new cluster ID: c6de14e62478410fb395f90c2b329ba7
I20260621 07:59:04.565475 25959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:59:04.583918 25959 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:59:04.585021 25959 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:59:04.590627 25959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Generated new TSK 0
I20260621 07:59:04.591364 25959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:59:04.626683 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44709
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:04.744148 25975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:04.744431 25975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:04.744524 25975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:04.748792 25975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:04.748857 25975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:04.748962 25975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:59:04.754299 25975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:04.755434 25975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:04.756543 25975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:59:04.766358 25975 server_base.cc:1034] running on GCE node
W20260621 07:59:04.766311 25983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:04.766300 25981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:04.766300 25980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:04.767030 25975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:04.767637 25975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:04.768826 25975 hybrid_clock.cc:648] HybridClock initialized: now 1782028744768807 us; error 32 us; skew 500 ppm
Jun 21 07:59:04 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:04.771543 25975 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:59:04.772498 25975 webserver.cc:460] Webserver started at http://127.11.223.129:36285/ using document root <none> and password file <none>
I20260621 07:59:04.773077 25975 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:04.773171 25975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:04.773416 25975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:04.775476 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "81c31cb0a0134413806b8a7b293c1dba"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "3f782d168bbb50fe7f053d71d74b22b3"
server_key_iv: "b10befa26ca11d32203fc6cb21e93043"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.776072 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "81c31cb0a0134413806b8a7b293c1dba"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "3f782d168bbb50fe7f053d71d74b22b3"
server_key_iv: "b10befa26ca11d32203fc6cb21e93043"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.780028 25975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:59:04.782529 25990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:04.783767 25975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 07:59:04.783915 25975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "81c31cb0a0134413806b8a7b293c1dba"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "3f782d168bbb50fe7f053d71d74b22b3"
server_key_iv: "b10befa26ca11d32203fc6cb21e93043"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.784046 25975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:04.794404 25975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:04.797266 25975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:04.797467 25975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:04.798049 25975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:04.798861 25975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:04.798936 25975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:04.799005 25975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:04.799057 25975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:04.807945 25975 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:33693
I20260621 07:59:04.807966 26103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:33693 every 8 connection(s)
I20260621 07:59:04.808899 25975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:59:04.813835 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 25975
I20260621 07:59:04.813994 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 07:59:04.814294 12158 external_mini_cluster.cc:1428] Setting key 1552073ca1917ad4552f175bfd610899
Jun 21 07:59:04 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:04.823479 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:04.810723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:42651 (local address 127.11.223.190:44709)
0621 07:59:04.810896 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:04.810900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:04.811753 (+ 853us) server_negotiation.cc:407] Connection header received
0621 07:59:04.813182 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:04.813185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:04.813241 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:04.813349 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:04.814654 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:04.815205 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:04.816350 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:04.816587 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:04.816711 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:04.819266 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:04.819272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:04.819276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:04.819308 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:04.821291 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:04.821811 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:04.821815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:04.821817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:04.821910 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:04.822249 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:04.822253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:04.822255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:04.822505 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:04.822633 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:04.823106 (+ 473us) server_negotiation.cc:299] Negotiation successful
0621 07:59:04.823246 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:59:04.824266 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:04.811113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:42651)
0621 07:59:04.811599 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:04.811642 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:04.812972 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:04.813525 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:04.813533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:04.813906 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:04.814515 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:04.814531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:04.815332 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:04.815336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:04.816205 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:04.816214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:04.816747 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:04.816750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:04.816911 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:04.817528 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:04.817544 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:04.819079 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:04.821423 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:04.821427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:04.821439 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:04.821701 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:04.822007 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:04.822009 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:04.822011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:04.822143 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:04.822621 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:04.822626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:04.822860 (+ 234us) client_negotiation.cc:769] Sending connection context
0621 07:59:04.823020 (+ 160us) client_negotiation.cc:240] Negotiation successful
0621 07:59:04.823272 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":111,"threads_started":1}
I20260621 07:59:04.825613 26104 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44709
I20260621 07:59:04.825920 26104 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 07:59:04.826493 26104 heartbeater.cc:511] Master 127.11.223.190:44709 requested a full tablet report, sending...
I20260621 07:59:04.828096 25913 ts_manager.cc:194] Registered new tserver with Master: 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693)
I20260621 07:59:04.829811 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:42651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:59:04.841233 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44709
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:04.949661 26109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:04.949929 26109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:04.949973 26109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:04.953920 26109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:04.953974 26109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:04.954031 26109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:59:04.959491 26109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:04.960546 26109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:04.961537 26109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:04.970811 26117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:04.970818 26114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:04.971058 26109 server_base.cc:1034] running on GCE node
W20260621 07:59:04.971199 26115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:04.971619 26109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:04.972303 26109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:04.973518 26109 hybrid_clock.cc:648] HybridClock initialized: now 1782028744973500 us; error 31 us; skew 500 ppm
Jun 21 07:59:04 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:04.976223 26109 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:59:04.977157 26109 webserver.cc:460] Webserver started at http://127.11.223.130:42651/ using document root <none> and password file <none>
I20260621 07:59:04.977720 26109 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:04.977782 26109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:04.977993 26109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:04.980020 26109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "5a6115eb2d9c45279c9f0995fe423ab8"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "415bfaa5591d9b33b8e106c7857c7972"
server_key_iv: "3ebf5fcbbb882035c9a3b6be2c362744"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.980602 26109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "5a6115eb2d9c45279c9f0995fe423ab8"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "415bfaa5591d9b33b8e106c7857c7972"
server_key_iv: "3ebf5fcbbb882035c9a3b6be2c362744"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.984163 26109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 07:59:04.986503 26124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:04.987879 26109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:04.988016 26109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "5a6115eb2d9c45279c9f0995fe423ab8"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "415bfaa5591d9b33b8e106c7857c7972"
server_key_iv: "3ebf5fcbbb882035c9a3b6be2c362744"
server_key_version: "encryptionkey@0"
I20260621 07:59:04.988142 26109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:05.012574 26109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:05.015781 26109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:05.015990 26109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:05.016654 26109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:05.017501 26109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:05.017581 26109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.017655 26109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:05.017719 26109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.026685 26109 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:35627
I20260621 07:59:05.026702 26237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:35627 every 8 connection(s)
I20260621 07:59:05.027683 26109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:05.037714 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26109
I20260621 07:59:05.037849 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:59:05.038141 12158 external_mini_cluster.cc:1428] Setting key 6b71d08f7337b11992cb2cedaf565358
I20260621 07:59:05.041734 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.029577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:54371 (local address 127.11.223.190:44709)
0621 07:59:05.029744 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.029749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.030626 (+ 877us) server_negotiation.cc:407] Connection header received
0621 07:59:05.032002 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.032005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.032054 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.032140 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:05.033458 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.033986 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.035129 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.035333 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.035407 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.038029 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:05.038032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:05.038034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:05.038067 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:05.039594 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.040124 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.040129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.040131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.040211 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.040604 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.040607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.040609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.040825 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:05.040932 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.041438 (+ 506us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.041572 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 07:59:05.042629 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.029992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:54371)
0621 07:59:05.030467 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.030514 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.031753 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.032413 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.032421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.032797 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:05.033312 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.033326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.034113 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.034116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.035024 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.035033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.035497 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.035500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.035645 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.036204 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:05.036220 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:05.037825 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:05.039724 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.039729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.039746 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.040013 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.040302 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.040306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.040308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.040485 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.040931 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:05.040938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:05.041192 (+ 254us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.041381 (+ 189us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.041725 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":113,"threads_started":1}
I20260621 07:59:05.043934 26238 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44709
I20260621 07:59:05.044191 26238 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:05.044718 26238 heartbeater.cc:511] Master 127.11.223.190:44709 requested a full tablet report, sending...
I20260621 07:59:05.045840 25913 ts_manager.cc:194] Registered new tserver with Master: 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
I20260621 07:59:05.046413 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:54371
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:59:05.066615 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44709
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:05.181906 26243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:05.182184 26243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:05.182227 26243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:05.186161 26243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:05.186225 26243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:05.186282 26243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:59:05.191581 26243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:05.192704 26243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:05.193679 26243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:59:05.203241 26243 server_base.cc:1034] running on GCE node
W20260621 07:59:05.203186 26248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:05.203140 26249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:05.203140 26251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:05.203922 26243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:05.204563 26243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:05.205725 26243 hybrid_clock.cc:648] HybridClock initialized: now 1782028745205693 us; error 45 us; skew 500 ppm
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:05.211710 26243 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:59:05.212735 26243 webserver.cc:460] Webserver started at http://127.11.223.131:41531/ using document root <none> and password file <none>
I20260621 07:59:05.213316 26243 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:05.213380 26243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:05.213629 26243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:05.215787 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "6ba7365107a740daad5244485909d03e"
format_stamp: "Formatted at 2026-06-21 07:59:05 on dist-test-slave-0pn3"
server_key: "09cedb377ffc53f8faf0a39a97622087"
server_key_iv: "88d18a47d0da60e5c014c121660b299d"
server_key_version: "encryptionkey@0"
I20260621 07:59:05.216434 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "6ba7365107a740daad5244485909d03e"
format_stamp: "Formatted at 2026-06-21 07:59:05 on dist-test-slave-0pn3"
server_key: "09cedb377ffc53f8faf0a39a97622087"
server_key_iv: "88d18a47d0da60e5c014c121660b299d"
server_key_version: "encryptionkey@0"
I20260621 07:59:05.220239 26243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260621 07:59:05.222808 26258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.224138 26243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:05.224236 26243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "6ba7365107a740daad5244485909d03e"
format_stamp: "Formatted at 2026-06-21 07:59:05 on dist-test-slave-0pn3"
server_key: "09cedb377ffc53f8faf0a39a97622087"
server_key_iv: "88d18a47d0da60e5c014c121660b299d"
server_key_version: "encryptionkey@0"
I20260621 07:59:05.224395 26243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:05.246618 26243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:05.249841 26243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:05.250015 26243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:05.250576 26243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:05.251408 26243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:05.251458 26243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.251499 26243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:05.251514 26243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.261260 26243 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:34731
I20260621 07:59:05.261277 26371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:34731 every 8 connection(s)
I20260621 07:59:05.262200 26243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:59:05.263325 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26243
I20260621 07:59:05.263423 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 07:59:05.263785 12158 external_mini_cluster.cc:1428] Setting key 23e4f11d55d679d2d0da89b0bd480aad
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:05.275882 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.264206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:32769 (local address 127.11.223.190:44709)
0621 07:59:05.264415 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.264420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.265474 (+ 1054us) server_negotiation.cc:407] Connection header received
0621 07:59:05.266960 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.266963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.267008 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.267076 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:05.268205 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.268921 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.270020 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.270223 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.270295 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.272818 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:05.272821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:05.272823 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:05.272847 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:05.274116 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.274571 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.274574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.274575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.274647 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.274963 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.274965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.274966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.275131 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:05.275204 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.275664 (+ 460us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.275757 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 07:59:05.276933 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.264684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:32769)
0621 07:59:05.265295 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.265369 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.266754 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.267221 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.267227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.267594 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:05.268086 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.268100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.269031 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.269034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.269921 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.269930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.270368 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.270370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.270578 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.271188 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:05.271205 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:05.272664 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:05.274217 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.274222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.274234 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.274486 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.274745 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.274747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.274748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.274884 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.275215 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:05.275219 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:05.275451 (+ 232us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.275621 (+ 170us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.275874 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":193,"threads_started":1}
I20260621 07:59:05.278013 26372 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44709
I20260621 07:59:05.278256 26372 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:05.278781 26372 heartbeater.cc:511] Master 127.11.223.190:44709 requested a full tablet report, sending...
I20260621 07:59:05.279798 25913 ts_manager.cc:194] Registered new tserver with Master: 6ba7365107a740daad5244485909d03e (127.11.223.131:34731)
I20260621 07:59:05.280452 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:32769
I20260621 07:59:05.290269 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:59:05.299319 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.292403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51748 (local address 127.11.223.190:44709)
0621 07:59:05.292559 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.292562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.292758 (+ 196us) server_negotiation.cc:407] Connection header received
0621 07:59:05.292878 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.292880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.292923 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.292989 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:05.293663 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.294153 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.294912 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.295131 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.295203 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.296632 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:05.296635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:05.296637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:05.296659 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:05.297874 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.298269 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.298271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.298272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.298347 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.298581 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.298583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.298584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.298791 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:05.298895 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.299109 (+ 214us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.299201 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260621 07:59:05.302429 25913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:59:05.305517 25913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:59:05.320540 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.314138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:33693 (local address 127.0.0.1:50574)
0621 07:59:05.314813 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.314873 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.315019 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.315943 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.315949 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.315973 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:05.316131 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.316138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.317755 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.317758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.318922 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.318932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.319935 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.319938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.320033 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.320060 (+ 27us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.320137 (+ 77us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.320332 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":101,"thread_start_us":96,"threads_started":1}
I20260621 07:59:05.320726 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.314980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.11.223.131:34731)
0621 07:59:05.315878 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.315891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.315912 (+ 21us) server_negotiation.cc:407] Connection header received
0621 07:59:05.315979 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.315983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.316145 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.316567 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:05.316714 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.317776 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.318888 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.319750 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.319900 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.320043 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.320483 (+ 440us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.320590 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":97,"threads_started":1}
I20260621 07:59:05.320729 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.315112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.11.223.130:35627)
0621 07:59:05.315432 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.315444 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.315640 (+ 196us) server_negotiation.cc:407] Connection header received
0621 07:59:05.315730 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.315736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.315934 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.316129 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:05.316302 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.317094 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.319258 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.319845 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.319968 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.320109 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.320483 (+ 374us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.320583 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":106,"threads_started":1}
I20260621 07:59:05.320735 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.314284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.11.223.129:33693)
0621 07:59:05.314614 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.314633 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.315424 (+ 791us) server_negotiation.cc:407] Connection header received
0621 07:59:05.315516 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.315521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.315757 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.316291 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:05.316465 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.317608 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.319046 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.319791 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.320274 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.320429 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.320482 (+ 53us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.320585 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":144,"threads_started":1}
I20260621 07:59:05.321262 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.314225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:34731 (local address 127.0.0.1:52394)
0621 07:59:05.315255 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.315270 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.315343 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.316290 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.316293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.316308 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:05.316421 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.316428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.317913 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.317916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.318757 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.318766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.319892 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.319898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.320011 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.320049 (+ 38us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.320163 (+ 114us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.320318 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":38,"thread_start_us":77,"threads_started":1}
I20260621 07:59:05.321740 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.314916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35627 (local address 127.0.0.1:56714)
0621 07:59:05.315536 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.315548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.315631 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.316089 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.316092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.316110 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:05.316205 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.316211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.317226 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.317233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.319143 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.319150 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.320119 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.320125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.320213 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.320236 (+ 23us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.320295 (+ 59us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.320388 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":19,"thread_start_us":91,"threads_started":1}
I20260621 07:59:05.323792 26038 tablet_service.cc:1467] Processing CreateTablet for tablet 59c38aa140bf46b99eb9d735d4401343 (DEFAULT_TABLE table=test-table [id=f1d74c33478c44a39afb1bb0a4a66f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:05.324071 26306 tablet_service.cc:1467] Processing CreateTablet for tablet 59c38aa140bf46b99eb9d735d4401343 (DEFAULT_TABLE table=test-table [id=f1d74c33478c44a39afb1bb0a4a66f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:05.324457 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 59c38aa140bf46b99eb9d735d4401343 (DEFAULT_TABLE table=test-table [id=f1d74c33478c44a39afb1bb0a4a66f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:05.324893 26038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c38aa140bf46b99eb9d735d4401343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:05.325204 26306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c38aa140bf46b99eb9d735d4401343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:05.325421 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c38aa140bf46b99eb9d735d4401343. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:05.331600 26389 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap starting.
I20260621 07:59:05.334553 26389 tablet_bootstrap.cc:654] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:05.334620 26391 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Bootstrap starting.
I20260621 07:59:05.335507 26389 log.cc:824] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:05.336268 26390 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Bootstrap starting.
I20260621 07:59:05.336939 26391 tablet_bootstrap.cc:654] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:05.338299 26391 log.cc:824] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:05.339087 26390 tablet_bootstrap.cc:654] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:05.339954 26390 log.cc:824] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:05.340695 26391 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: No bootstrap required, opened a new log
I20260621 07:59:05.340900 26391 ts_tablet_manager.cc:1397] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260621 07:59:05.341771 26389 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: No bootstrap required, opened a new log
I20260621 07:59:05.341814 26390 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: No bootstrap required, opened a new log
I20260621 07:59:05.341980 26389 ts_tablet_manager.cc:1397] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260621 07:59:05.342072 26390 ts_tablet_manager.cc:1397] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260621 07:59:05.345064 26391 raft_consensus.cc:348] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.345062 26390 raft_consensus.cc:348] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.345356 26390 raft_consensus.cc:374] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:05.345443 26390 raft_consensus.cc:729] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6115eb2d9c45279c9f0995fe423ab8, State: Initialized, Role: FOLLOWER
I20260621 07:59:05.345489 26391 raft_consensus.cc:374] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:05.345575 26391 raft_consensus.cc:729] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7365107a740daad5244485909d03e, State: Initialized, Role: FOLLOWER
I20260621 07:59:05.345952 26390 consensus_queue.cc:260] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.346006 26391 consensus_queue.cc:260] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.346249 26389 raft_consensus.cc:348] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.346777 26238 heartbeater.cc:503] Master 127.11.223.190:44709 was elected leader, sending a full tablet report...
I20260621 07:59:05.346987 26390 ts_tablet_manager.cc:1428] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260621 07:59:05.347012 26391 ts_tablet_manager.cc:1428] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260621 07:59:05.347323 26372 heartbeater.cc:503] Master 127.11.223.190:44709 was elected leader, sending a full tablet report...
I20260621 07:59:05.347784 26389 raft_consensus.cc:374] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:05.347874 26389 raft_consensus.cc:729] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c31cb0a0134413806b8a7b293c1dba, State: Initialized, Role: FOLLOWER
I20260621 07:59:05.348392 26389 consensus_queue.cc:260] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.349277 26104 heartbeater.cc:503] Master 127.11.223.190:44709 was elected leader, sending a full tablet report...
I20260621 07:59:05.349481 26389 ts_tablet_manager.cc:1428] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260621 07:59:05.427405 26396 raft_consensus.cc:482] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:05.427659 26396 raft_consensus.cc:504] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.428936 26396 leader_election.cc:288] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693), 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
I20260621 07:59:05.432687 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.429342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:38515 (local address 127.11.223.129:33693)
0621 07:59:05.429478 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.429489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.429507 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:59:05.429568 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.429574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.429643 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.429732 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:05.430085 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.430610 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.431661 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.432079 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.432210 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.432269 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.432490 (+ 221us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.432562 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:59:05.432686 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.429197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:33693 (local address 127.11.223.131:38515)
0621 07:59:05.429335 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.429349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.429462 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.429818 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.429824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.429845 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:05.429965 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.429970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.430723 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.430729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.431533 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.431541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.432198 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.432201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.432343 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.432377 (+ 34us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.432475 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.432550 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260621 07:59:05.433084 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.429659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:55095 (local address 127.11.223.130:35627)
0621 07:59:05.429802 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.429807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.430000 (+ 193us) server_negotiation.cc:407] Connection header received
0621 07:59:05.430144 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.430147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.430199 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.430303 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:05.430563 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.431044 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.432069 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.432504 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.432635 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.432681 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.432883 (+ 202us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.432966 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260621 07:59:05.433427 26398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.429494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35627 (local address 127.11.223.131:55095)
0621 07:59:05.429875 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.429917 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.430032 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.430311 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.430314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.430329 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:05.430451 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.430457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.431160 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.431166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.431942 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.431950 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.432628 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.432632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.432726 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.432751 (+ 25us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.433235 (+ 484us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.433313 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":153,"threads_started":1}
I20260621 07:59:05.433393 26058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c38aa140bf46b99eb9d735d4401343" candidate_uuid: "6ba7365107a740daad5244485909d03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c31cb0a0134413806b8a7b293c1dba" is_pre_election: true
I20260621 07:59:05.433702 26058 raft_consensus.cc:2454] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7365107a740daad5244485909d03e in term 0.
I20260621 07:59:05.434121 26192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c38aa140bf46b99eb9d735d4401343" candidate_uuid: "6ba7365107a740daad5244485909d03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" is_pre_election: true
I20260621 07:59:05.434190 26261 leader_election.cc:302] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba7365107a740daad5244485909d03e, 81c31cb0a0134413806b8a7b293c1dba; no voters:
I20260621 07:59:05.434412 26192 raft_consensus.cc:2454] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7365107a740daad5244485909d03e in term 0.
I20260621 07:59:05.434456 26396 raft_consensus.cc:2781] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:59:05.434569 26396 raft_consensus.cc:482] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:59:05.434626 26396 raft_consensus.cc:3037] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:05.435725 26396 raft_consensus.cc:504] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.436103 26396 leader_election.cc:288] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [CANDIDATE]: Term 1 election: Requested vote from peers 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693), 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
I20260621 07:59:05.436539 26058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c38aa140bf46b99eb9d735d4401343" candidate_uuid: "6ba7365107a740daad5244485909d03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c31cb0a0134413806b8a7b293c1dba"
I20260621 07:59:05.436556 26192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c38aa140bf46b99eb9d735d4401343" candidate_uuid: "6ba7365107a740daad5244485909d03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6115eb2d9c45279c9f0995fe423ab8"
I20260621 07:59:05.436669 26192 raft_consensus.cc:3037] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:05.436669 26058 raft_consensus.cc:3037] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:05.437963 26192 raft_consensus.cc:2454] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7365107a740daad5244485909d03e in term 1.
I20260621 07:59:05.437963 26058 raft_consensus.cc:2454] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7365107a740daad5244485909d03e in term 1.
I20260621 07:59:05.438241 26261 leader_election.cc:302] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba7365107a740daad5244485909d03e, 81c31cb0a0134413806b8a7b293c1dba; no voters:
I20260621 07:59:05.438462 26396 raft_consensus.cc:2781] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:05.438690 26396 raft_consensus.cc:686] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 1 LEADER]: Becoming Leader. State: Replica: 6ba7365107a740daad5244485909d03e, State: Running, Role: LEADER
I20260621 07:59:05.439038 26396 consensus_queue.cc:237] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.442198 25913 catalog_manager.cc:5495] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ba7365107a740daad5244485909d03e (127.11.223.131). New cstate: current_term: 1 leader_uuid: "6ba7365107a740daad5244485909d03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } health_report { overall_health: UNKNOWN } } }
I20260621 07:59:05.481423 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.478114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.11.223.131:34731)
0621 07:59:05.478243 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.478247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.478286 (+ 39us) server_negotiation.cc:407] Connection header received
0621 07:59:05.478364 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.478367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.478421 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.478509 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:59:05.478733 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.479245 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.480038 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.480242 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.480385 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.480636 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:59:05.481060 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:59:05.481161 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.481223 (+ 62us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.481273 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 07:59:05.487349 26058 raft_consensus.cc:1261] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Refusing update from remote peer 6ba7365107a740daad5244485909d03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:05.487396 26192 raft_consensus.cc:1261] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7365107a740daad5244485909d03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:05.488148 26396 consensus_queue.cc:1035] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:05.488366 26399 consensus_queue.cc:1035] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:05.495227 26402 mvcc.cc:205] Tried to move back new op lower bound from 7299189741508669440 to 7299189741321064448. Current Snapshot: MvccSnapshot[applied={T|T < 7299189741508669440}]
I20260621 07:59:05.495631 26403 mvcc.cc:205] Tried to move back new op lower bound from 7299189741508669440 to 7299189741321064448. Current Snapshot: MvccSnapshot[applied={T|T < 7299189741508669440}]
I20260621 07:59:05.505352 25908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:05.505510 25908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:05.508005 25908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Sending DeleteTablet for 3 replicas of tablet 59c38aa140bf46b99eb9d735d4401343
I20260621 07:59:05.508922 26172 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c38aa140bf46b99eb9d735d4401343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:05 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:56714
I20260621 07:59:05.508898 26038 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c38aa140bf46b99eb9d735d4401343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:05 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50574
I20260621 07:59:05.509429 26412 tablet_replica.cc:330] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: stopping tablet replica
I20260621 07:59:05.509433 26306 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c38aa140bf46b99eb9d735d4401343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:05 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:52394
I20260621 07:59:05.509781 26412 raft_consensus.cc:2229] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:05.509887 26413 tablet_replica.cc:330] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: stopping tablet replica
I20260621 07:59:05.510035 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25975
I20260621 07:59:05.510203 26412 raft_consensus.cc:2258] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:05.510218 26413 raft_consensus.cc:2229] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:59:05.510617 26413 raft_consensus.cc:2258] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:05.510762 26411 tablet_replica.cc:330] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: stopping tablet replica
I20260621 07:59:05.511075 26411 raft_consensus.cc:2229] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:05.511348 26411 raft_consensus.cc:2258] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:05.513000 26411 ts_tablet_manager.cc:1905] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:05.513161 26413 ts_tablet_manager.cc:1905] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:05.515815 26413 ts_tablet_manager.cc:1918] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:05.516062 26413 log.cc:1196] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/59c38aa140bf46b99eb9d735d4401343
I20260621 07:59:05.516388 26411 ts_tablet_manager.cc:1918] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:05.516440 26413 ts_tablet_manager.cc:1939] T 59c38aa140bf46b99eb9d735d4401343 P 6ba7365107a740daad5244485909d03e: Deleting consensus metadata
I20260621 07:59:05.516526 26411 log.cc:1196] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/59c38aa140bf46b99eb9d735d4401343
I20260621 07:59:05.516849 26411 ts_tablet_manager.cc:1939] T 59c38aa140bf46b99eb9d735d4401343 P 5a6115eb2d9c45279c9f0995fe423ab8: Deleting consensus metadata
I20260621 07:59:05.517653 25900 catalog_manager.cc:4841] TS 6ba7365107a740daad5244485909d03e (127.11.223.131:34731): tablet 59c38aa140bf46b99eb9d735d4401343 (table test-table [id=f1d74c33478c44a39afb1bb0a4a66f53]) successfully deleted
I20260621 07:59:05.518038 25899 catalog_manager.cc:4841] TS 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627): tablet 59c38aa140bf46b99eb9d735d4401343 (table test-table [id=f1d74c33478c44a39afb1bb0a4a66f53]) successfully deleted
I20260621 07:59:05.519173 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 26109
W20260621 07:59:05.519182 25899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:33693 (error 108)
I20260621 07:59:05.525199 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 26243
I20260621 07:59:05.525450 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.525158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:33693 (local address 127.0.0.1:50588)
0621 07:59:05.525271 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.525342 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:33693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260621 07:59:05.525838 25899 catalog_manager.cc:4584] TS 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59c38aa140bf46b99eb9d735d4401343: Network error: Client connection negotiation failed: client connection to 127.11.223.129:33693: connect: Connection refused (error 111)
I20260621 07:59:05.531033 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 25881
I20260621 07:59:05.538048 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44709
--webserver_interface=127.11.223.190
--webserver_port=40659
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:05.651294 26415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:05.651593 26415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:05.651635 26415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:05.655736 26415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:05.655804 26415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:05.655824 26415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:05.655841 26415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:05.655897 26415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:05.662055 26415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=40659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:05.663156 26415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:05.664139 26415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:05.677112 26424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:05.677112 26421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:05.677127 26422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:05.677502 26415 server_base.cc:1034] running on GCE node
I20260621 07:59:05.678118 26415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:05.679279 26415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:05.680408 26415 hybrid_clock.cc:648] HybridClock initialized: now 1782028745680365 us; error 55 us; skew 500 ppm
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:05.683192 26415 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:05.684124 26415 webserver.cc:460] Webserver started at http://127.11.223.190:40659/ using document root <none> and password file <none>
I20260621 07:59:05.684741 26415 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:05.684816 26415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:05.688630 26415 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260621 07:59:05.690794 26431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.692176 26415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:05.692286 26415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ff2712f0acd6425ca0192b0a11d6d6c6"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "4bbd69095542d333d79e1f06f9db6ca7"
server_key_iv: "fe83e515e0577d7c2eee5f388c41a0ce"
server_key_version: "encryptionkey@0"
I20260621 07:59:05.692929 26415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:05.705125 26415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:05.707970 26415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:05.708125 26415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:05.715062 26415 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:44709
I20260621 07:59:05.715349 26483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:44709 every 8 connection(s)
I20260621 07:59:05.716190 26415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:05.720582 26484 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:59:05.722219 26484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap starting.
I20260621 07:59:05.724681 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26415
I20260621 07:59:05.725119 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:33693
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:44709
--webserver_port=36285
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:05.740237 26484 log.cc:824] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:05.749531 26484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:59:05.750056 26484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap complete.
I20260621 07:59:05.754271 26484 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:05.754719 26484 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2712f0acd6425ca0192b0a11d6d6c6, State: Initialized, Role: FOLLOWER
I20260621 07:59:05.755256 26484 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:05.755399 26484 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:59:05.755494 26484 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:59:05.755609 26484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:59:05.757619 26484 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:05.757965 26484 leader_election.cc:302] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff2712f0acd6425ca0192b0a11d6d6c6; no voters:
I20260621 07:59:05.758332 26484 leader_election.cc:288] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:59:05.758433 26488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:59:05.758641 26488 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 LEADER]: Becoming Leader. State: Replica: ff2712f0acd6425ca0192b0a11d6d6c6, State: Running, Role: LEADER
I20260621 07:59:05.758991 26488 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:05.759588 26484 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:05.760454 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } } }
I20260621 07:59:05.760576 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:05.760963 26490 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff2712f0acd6425ca0192b0a11d6d6c6. Latest consensus state: current_term: 2 leader_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } } }
I20260621 07:59:05.761044 26490 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:05.764755 26504 catalog_manager.cc:1259] Loaded cluster ID: c6de14e62478410fb395f90c2b329ba7
I20260621 07:59:05.764817 26504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: loading cluster ID for follower catalog manager: success
W20260621 07:59:05.765759 26504 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260621 07:59:05.765833 26504 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260621 07:59:05.766002 26497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:05.768424 26497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:05.768903 26497 catalog_manager.cc:1259] Loaded cluster ID: c6de14e62478410fb395f90c2b329ba7
I20260621 07:59:05.768962 26497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:59:05.770089 26497 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260621 07:59:05.774695 26497 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260621 07:59:05.780124 26497 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:59:05.780965 26497 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Loaded TSK: 0
I20260621 07:59:05.782137 26497 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 07:59:05.843550 26486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:05.843818 26486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:05.843869 26486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:05.847970 26486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:05.848030 26486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:05.848091 26486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:59:05.854210 26486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:33693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=36285
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:05.855221 26486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:05.856254 26486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:05.865985 26512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:05.866003 26514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:05.866356 26486 server_base.cc:1034] running on GCE node
W20260621 07:59:05.865988 26511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:05.866782 26486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:05.867482 26486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:05.868651 26486 hybrid_clock.cc:648] HybridClock initialized: now 1782028745868626 us; error 46 us; skew 500 ppm
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:05.871446 26486 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:59:05.872465 26486 webserver.cc:460] Webserver started at http://127.11.223.129:36285/ using document root <none> and password file <none>
I20260621 07:59:05.873057 26486 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:05.873159 26486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:05.877056 26486 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260621 07:59:05.879256 26521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:05.880662 26486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:59:05.880755 26486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "81c31cb0a0134413806b8a7b293c1dba"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "3f782d168bbb50fe7f053d71d74b22b3"
server_key_iv: "b10befa26ca11d32203fc6cb21e93043"
server_key_version: "encryptionkey@0"
I20260621 07:59:05.881318 26486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:05.895311 26486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:05.898665 26486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:05.898880 26486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:05.899542 26486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:05.900810 26528 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260621 07:59:05.905692 26486 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 07:59:05.905781 26486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260621 07:59:05.905833 26486 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 07:59:05.907631 26486 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 07:59:05.907680 26486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260621 07:59:05.907990 26528 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap starting.
I20260621 07:59:05.918673 26486 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:33693
I20260621 07:59:05.918773 26635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:33693 every 8 connection(s)
I20260621 07:59:05.919684 26486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:59:05.922268 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26486
I20260621 07:59:05.922875 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:35627
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:44709
--webserver_port=42651
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:05.926697 26528 log.cc:824] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Log is configured to *not* fsync() on all Append() calls
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:05.936823 26528 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:59:05.937131 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.921825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:38163 (local address 127.11.223.190:44709)
0621 07:59:05.922865 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.922897 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.922943 (+ 46us) server_negotiation.cc:407] Connection header received
0621 07:59:05.923990 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.924002 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.924356 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.924753 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:05.926141 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.927390 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.928707 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.929210 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.929402 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.931867 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:05.931872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:05.931884 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:05.931914 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:05.933750 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.934318 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.934323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.934327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.934453 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.934890 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.934894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.934895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.935290 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:05.935437 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.935978 (+ 541us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.936242 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":130,"threads_started":1}
I20260621 07:59:05.937316 26528 tablet_bootstrap.cc:492] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap complete.
I20260621 07:59:05.937574 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.921678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:38163)
0621 07:59:05.922444 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.922498 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.923554 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.924868 (+ 1314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.924878 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.925346 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:05.925957 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.925977 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.927564 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.927568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.928580 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.928590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.929376 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.929380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.929579 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.930199 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:05.930216 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:05.931660 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:05.933898 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.933904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.933917 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.934186 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.934560 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.934563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.934565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.934755 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.935416 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:05.935422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:05.935656 (+ 234us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.935839 (+ 183us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.936182 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":189,"threads_started":1}
I20260621 07:59:05.938735 26528 ts_tablet_manager.cc:1397] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.012s
I20260621 07:59:05.939930 26636 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44709
I20260621 07:59:05.940161 26636 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:05.940696 26636 heartbeater.cc:511] Master 127.11.223.190:44709 requested a full tablet report, sending...
I20260621 07:59:05.941687 26528 raft_consensus.cc:348] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.942198 26528 raft_consensus.cc:729] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c31cb0a0134413806b8a7b293c1dba, State: Initialized, Role: FOLLOWER
I20260621 07:59:05.942665 26528 consensus_queue.cc:260] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } } peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } }
I20260621 07:59:05.943195 26448 ts_manager.cc:194] Registered new tserver with Master: 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693)
I20260621 07:59:05.943691 26528 ts_tablet_manager.cc:1428] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260621 07:59:05.946864 26448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:38163
I20260621 07:59:05.949527 26636 heartbeater.cc:503] Master 127.11.223.190:44709 was elected leader, sending a full tablet report...
Jun 21 07:59:05 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:59:05.957751 26644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.946052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.11.223.129:33693)
0621 07:59:05.947672 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:05.947682 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:05.947701 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:59:05.947753 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:05.947757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:05.947967 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:05.948114 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:05.948931 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.949948 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.951183 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.951704 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.951857 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.954184 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:05.954192 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:05.954204 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:05.954234 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:05.955824 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.956363 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.956371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.956373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.956458 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:05.956739 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:05.956743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:05.956744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:05.956983 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:05.957123 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:05.957445 (+ 322us) server_negotiation.cc:299] Negotiation successful
0621 07:59:05.957602 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":1523,"thread_start_us":80,"threads_started":1}
I20260621 07:59:05.957746 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:05.945973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:33693 (local address 127.0.0.1:50592)
0621 07:59:05.946383 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:05.946434 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:05.946561 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:05.948148 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:05.948154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:05.948642 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:05.948781 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.948793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.950096 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.950099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.951063 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:05.951072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:05.951857 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:05.951861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:05.951978 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:05.952676 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:05.952689 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:05.954048 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:05.955970 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.955979 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.955984 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.956223 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.956556 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:05.956560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:05.956562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:05.956644 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:05.957096 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:05.957101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:05.957224 (+ 123us) client_negotiation.cc:769] Sending connection context
0621 07:59:05.957357 (+ 133us) client_negotiation.cc:240] Negotiation successful
0621 07:59:05.957561 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":112,"threads_started":1}
I20260621 07:59:05.958738 26570 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c38aa140bf46b99eb9d735d4401343 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:05 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50592
I20260621 07:59:05.959774 26645 tablet_replica.cc:330] stopping tablet replica
I20260621 07:59:05.960074 26645 raft_consensus.cc:2229] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:05.960474 26645 raft_consensus.cc:2258] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:05.961490 26645 ts_tablet_manager.cc:1905] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:05.965068 26645 ts_tablet_manager.cc:1918] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:05.965148 26645 log.cc:1196] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/59c38aa140bf46b99eb9d735d4401343
I20260621 07:59:05.965432 26645 ts_tablet_manager.cc:1939] T 59c38aa140bf46b99eb9d735d4401343 P 81c31cb0a0134413806b8a7b293c1dba: Deleting consensus metadata
I20260621 07:59:05.967139 26434 catalog_manager.cc:4841] TS 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693): tablet 59c38aa140bf46b99eb9d735d4401343 (table test-table [id=f1d74c33478c44a39afb1bb0a4a66f53]) successfully deleted
W20260621 07:59:06.046983 26640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:06.047293 26640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:06.047339 26640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:06.051671 26640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:06.051743 26640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:06.051806 26640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:59:06.057238 26640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:35627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=42651
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:06.058274 26640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:06.059278 26640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:06.068883 26651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.068931 26650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.068883 26653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:06.069128 26640 server_base.cc:1034] running on GCE node
I20260621 07:59:06.069650 26640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:06.070254 26640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:06.071414 26640 hybrid_clock.cc:648] HybridClock initialized: now 1782028746071400 us; error 26 us; skew 500 ppm
Jun 21 07:59:06 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:06.074385 26640 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:59:06.075266 26640 webserver.cc:460] Webserver started at http://127.11.223.130:42651/ using document root <none> and password file <none>
I20260621 07:59:06.075847 26640 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:06.075958 26640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:06.079926 26640 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:59:06.082267 26660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.083745 26640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260621 07:59:06.083881 26640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "5a6115eb2d9c45279c9f0995fe423ab8"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "415bfaa5591d9b33b8e106c7857c7972"
server_key_iv: "3ebf5fcbbb882035c9a3b6be2c362744"
server_key_version: "encryptionkey@0"
I20260621 07:59:06.084523 26640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:06.101580 26640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:06.104717 26640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:06.104938 26640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:06.105559 26640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:06.106419 26640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:06.106493 26640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.106570 26640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:06.106613 26640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.116854 26640 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:35627
I20260621 07:59:06.116873 26773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:35627 every 8 connection(s)
I20260621 07:59:06.117902 26640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:59:06.121999 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26640
I20260621 07:59:06.122514 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:34731
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:44709
--webserver_port=41531
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:59:06 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:06.134143 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.119750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:48341 (local address 127.11.223.190:44709)
0621 07:59:06.119894 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.119897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.120732 (+ 835us) server_negotiation.cc:407] Connection header received
0621 07:59:06.122259 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.122262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.122316 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.122396 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:06.124100 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.124994 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.126144 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.126425 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.126508 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.129905 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:06.129908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:06.129911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:06.129937 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:06.131512 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:06.132219 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:06.132222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:06.132223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:06.132303 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:06.132838 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:06.132840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:06.132842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:06.133059 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:06.133137 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.133872 (+ 735us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.134001 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:59:06.135561 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.120077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:48341)
0621 07:59:06.120579 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.120622 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:06.121997 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:06.122608 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:06.122620 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:06.123205 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:06.123906 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.123928 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.125131 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.125133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.126017 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.126026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.126591 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.126597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.126838 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.127519 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:06.127545 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:06.129586 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:06.131643 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:06.131653 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:06.131673 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:06.132082 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:06.132484 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:06.132489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:06.132492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:06.132713 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:06.133172 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:06.133182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:06.133575 (+ 393us) client_negotiation.cc:769] Sending connection context
0621 07:59:06.133837 (+ 262us) client_negotiation.cc:240] Negotiation successful
0621 07:59:06.134274 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":109,"threads_started":1}
I20260621 07:59:06.136925 26774 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44709
I20260621 07:59:06.137176 26774 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:06.137673 26774 heartbeater.cc:511] Master 127.11.223.190:44709 requested a full tablet report, sending...
I20260621 07:59:06.138756 26448 ts_manager.cc:194] Registered new tserver with Master: 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
I20260621 07:59:06.139322 26448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:48341
W20260621 07:59:06.237063 26777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:06.237324 26777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:06.237362 26777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:06.241444 26777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:06.241541 26777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:06.241634 26777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:59:06.246925 26777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:34731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=41531
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:06.248008 26777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:06.249097 26777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:59:06.258713 26777 server_base.cc:1034] running on GCE node
W20260621 07:59:06.258615 26782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.258615 26783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.258662 26785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:06.259358 26777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:06.260048 26777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:06.261226 26777 hybrid_clock.cc:648] HybridClock initialized: now 1782028746261193 us; error 77 us; skew 500 ppm
Jun 21 07:59:06 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:06.264010 26777 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:59:06.264892 26777 webserver.cc:460] Webserver started at http://127.11.223.131:41531/ using document root <none> and password file <none>
I20260621 07:59:06.265439 26777 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:06.265511 26777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:06.269274 26777 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:59:06.271302 26792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.272559 26777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:59:06.272702 26777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "6ba7365107a740daad5244485909d03e"
format_stamp: "Formatted at 2026-06-21 07:59:05 on dist-test-slave-0pn3"
server_key: "09cedb377ffc53f8faf0a39a97622087"
server_key_iv: "88d18a47d0da60e5c014c121660b299d"
server_key_version: "encryptionkey@0"
I20260621 07:59:06.273254 26777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:06.294621 26777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:06.298084 26777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:06.298297 26777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:06.298906 26777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:06.299729 26777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:06.299803 26777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.299902 26777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:06.299954 26777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.308480 26777 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:34731
I20260621 07:59:06.308506 26905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:34731 every 8 connection(s)
I20260621 07:59:06.309471 26777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:59:06.310863 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26777
Jun 21 07:59:06 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:06.319957 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.311494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.11.223.190:44709)
0621 07:59:06.311741 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.311746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.311998 (+ 252us) server_negotiation.cc:407] Connection header received
0621 07:59:06.312172 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.312175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.312231 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.312346 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:06.313103 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.313648 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.314623 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.314863 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.314979 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.316403 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:06.316409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:06.316417 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:06.316439 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:06.318081 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:06.318506 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:06.318509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:06.318511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:06.318619 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:06.318922 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:06.318925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:06.318927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:06.319138 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:06.319359 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.319639 (+ 280us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.319767 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260621 07:59:06.323261 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.311357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:39945 (local address 127.11.223.190:44709)
0621 07:59:06.311522 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.311528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.312483 (+ 955us) server_negotiation.cc:407] Connection header received
0621 07:59:06.313852 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.313856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.313902 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.313977 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:06.315096 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.315762 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.316975 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.317192 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.317311 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.319848 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:06.319852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:06.319854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:06.319876 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:06.321438 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:06.321959 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:06.321961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:06.321963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:06.322068 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:06.322372 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:06.322374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:06.322375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:06.322522 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:06.322594 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.323055 (+ 461us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.323148 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260621 07:59:06.324172 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.311850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:39945)
0621 07:59:06.312302 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.312402 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:06.313672 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:06.314138 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:06.314145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:06.314489 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:06.314950 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.314962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.315910 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.315913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.316849 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.316858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.317360 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.317364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.317532 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.318058 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:06.318079 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:06.319671 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:06.321551 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:06.321556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:06.321568 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:06.321838 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:06.322154 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:06.322156 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:06.322158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:06.322287 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:06.322606 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:06.322610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:06.322837 (+ 227us) client_negotiation.cc:769] Sending connection context
0621 07:59:06.323010 (+ 173us) client_negotiation.cc:240] Negotiation successful
0621 07:59:06.323255 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":134,"threads_started":1}
I20260621 07:59:06.325307 26906 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44709
I20260621 07:59:06.325546 26906 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:06.326097 26906 heartbeater.cc:511] Master 127.11.223.190:44709 requested a full tablet report, sending...
I20260621 07:59:06.327111 26448 ts_manager.cc:194] Registered new tserver with Master: 6ba7365107a740daad5244485909d03e (127.11.223.131:34731)
I20260621 07:59:06.327667 26448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:39945
I20260621 07:59:06.333778 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:59:06.338115 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.334426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51760 (local address 127.11.223.190:44709)
0621 07:59:06.334571 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.334576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.334825 (+ 249us) server_negotiation.cc:407] Connection header received
0621 07:59:06.334950 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.334952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.335053 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.335137 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:59:06.335340 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.335957 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.336855 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.337052 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.337126 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.337389 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:59:06.337766 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:59:06.337871 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.337899 (+ 28us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.337945 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 07:59:06.338764 26448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:59:06.342095 26448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:59:06.350289 26570 tablet_service.cc:1467] Processing CreateTablet for tablet 09695f95850a402793a91ec831c0603e (DEFAULT_TABLE table=test-table [id=4c5fea5bee084a0cb30d8fb522459f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:06.350947 26570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09695f95850a402793a91ec831c0603e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:06.354022 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.348550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:34731 (local address 127.0.0.1:52404)
0621 07:59:06.348716 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.348731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:06.348806 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:06.349629 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:06.349632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:06.349662 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:06.349774 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.349784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.351326 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.351329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.352618 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.352630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.353531 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.353535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.353627 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.353668 (+ 41us) client_negotiation.cc:769] Sending connection context
0621 07:59:06.353787 (+ 119us) client_negotiation.cc:240] Negotiation successful
0621 07:59:06.353877 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":39}
I20260621 07:59:06.354035 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.348756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.11.223.131:34731)
0621 07:59:06.349045 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.349059 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.349085 (+ 26us) server_negotiation.cc:407] Connection header received
0621 07:59:06.349171 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.349180 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.349450 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.349943 (+ 493us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:06.350112 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.351163 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.352750 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.353402 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.353536 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.353671 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.353818 (+ 147us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.353909 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":117,"threads_started":1}
I20260621 07:59:06.354040 26528 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap starting.
I20260621 07:59:06.354307 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.349275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35627 (local address 127.0.0.1:56726)
0621 07:59:06.349522 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.349535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:06.349626 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:06.350259 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:06.350262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:06.350278 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:06.350372 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.350378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.351862 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.351868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.352776 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.352785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.353928 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.353934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.354032 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.354064 (+ 32us) client_negotiation.cc:769] Sending connection context
0621 07:59:06.354117 (+ 53us) client_negotiation.cc:240] Negotiation successful
0621 07:59:06.354185 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":31,"thread_start_us":90,"threads_started":1}
I20260621 07:59:06.354921 26528 tablet_bootstrap.cc:654] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:06.354965 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.349446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.11.223.130:35627)
0621 07:59:06.349863 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.349871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.349890 (+ 19us) server_negotiation.cc:407] Connection header received
0621 07:59:06.349953 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.349956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.350119 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.350286 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:06.350455 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.351741 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.352908 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.353786 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.354510 (+ 724us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.354648 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.354696 (+ 48us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.354780 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":145,"threads_started":1}
I20260621 07:59:06.356662 26528 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: No bootstrap required, opened a new log
I20260621 07:59:06.356756 26528 ts_tablet_manager.cc:1397] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260621 07:59:06.357213 26528 raft_consensus.cc:348] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.357338 26528 raft_consensus.cc:374] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:06.357378 26528 raft_consensus.cc:729] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c31cb0a0134413806b8a7b293c1dba, State: Initialized, Role: FOLLOWER
I20260621 07:59:06.357589 26528 consensus_queue.cc:260] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.357368 26840 tablet_service.cc:1467] Processing CreateTablet for tablet 09695f95850a402793a91ec831c0603e (DEFAULT_TABLE table=test-table [id=4c5fea5bee084a0cb30d8fb522459f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:06.357874 26528 ts_tablet_manager.cc:1428] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:59:06.357784 26708 tablet_service.cc:1467] Processing CreateTablet for tablet 09695f95850a402793a91ec831c0603e (DEFAULT_TABLE table=test-table [id=4c5fea5bee084a0cb30d8fb522459f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:06.358556 26840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09695f95850a402793a91ec831c0603e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:06.358867 26708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09695f95850a402793a91ec831c0603e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:06.365734 26919 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Bootstrap starting.
I20260621 07:59:06.368207 26919 tablet_bootstrap.cc:654] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:06.368705 26920 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Bootstrap starting.
I20260621 07:59:06.369073 26919 log.cc:824] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:06.370853 26920 tablet_bootstrap.cc:654] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:06.370858 26919 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: No bootstrap required, opened a new log
I20260621 07:59:06.371065 26919 ts_tablet_manager.cc:1397] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:59:06.371611 26920 log.cc:824] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:06.373404 26920 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: No bootstrap required, opened a new log
I20260621 07:59:06.373629 26920 ts_tablet_manager.cc:1397] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 07:59:06.374447 26919 raft_consensus.cc:348] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.374687 26919 raft_consensus.cc:374] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:06.374766 26919 raft_consensus.cc:729] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6115eb2d9c45279c9f0995fe423ab8, State: Initialized, Role: FOLLOWER
I20260621 07:59:06.375257 26919 consensus_queue.cc:260] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.375977 26774 heartbeater.cc:503] Master 127.11.223.190:44709 was elected leader, sending a full tablet report...
I20260621 07:59:06.376261 26919 ts_tablet_manager.cc:1428] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 07:59:06.377267 26920 raft_consensus.cc:348] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.377532 26920 raft_consensus.cc:374] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:06.377599 26920 raft_consensus.cc:729] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7365107a740daad5244485909d03e, State: Initialized, Role: FOLLOWER
I20260621 07:59:06.378026 26920 consensus_queue.cc:260] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.378748 26906 heartbeater.cc:503] Master 127.11.223.190:44709 was elected leader, sending a full tablet report...
I20260621 07:59:06.379009 26920 ts_tablet_manager.cc:1428] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260621 07:59:06.421060 26637 tablet.cc:2369] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:59:06.510195 26923 raft_consensus.cc:482] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:06.510425 26923 raft_consensus.cc:504] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.511583 26923 leader_election.cc:288] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ba7365107a740daad5244485909d03e (127.11.223.131:34731), 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693)
I20260621 07:59:06.516101 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.511933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:34339 (local address 127.11.223.131:34731)
0621 07:59:06.512092 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.512096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.512116 (+ 20us) server_negotiation.cc:407] Connection header received
0621 07:59:06.512233 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.512238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.512297 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.512382 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:06.512711 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.513553 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.514967 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.515507 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.515628 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.515688 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.515884 (+ 196us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.515951 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260621 07:59:06.516103 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.511962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:47299 (local address 127.11.223.129:33693)
0621 07:59:06.512225 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.512229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.512440 (+ 211us) server_negotiation.cc:407] Connection header received
0621 07:59:06.512566 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.512568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.512618 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.512751 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:06.512993 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.513751 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.514998 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.515599 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.515693 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.515864 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.515896 (+ 32us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.515953 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":100,"threads_started":1}
I20260621 07:59:06.516125 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.511807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:34731 (local address 127.11.223.130:34339)
0621 07:59:06.512010 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.512026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:06.512128 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:06.512411 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:06.512415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:06.512453 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:06.512595 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.512601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.513657 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.513659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.514824 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.514836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.515620 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.515623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.515741 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.515766 (+ 25us) client_negotiation.cc:769] Sending connection context
0621 07:59:06.515843 (+ 77us) client_negotiation.cc:240] Negotiation successful
0621 07:59:06.515950 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":16,"spinlock_wait_cycles":7040}
I20260621 07:59:06.516132 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.511845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:33693 (local address 127.11.223.130:47299)
0621 07:59:06.512290 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.512301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:06.512454 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:06.512771 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:06.512773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:06.512787 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:06.512884 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.512888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.513867 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.513870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.514897 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:06.514906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.515693 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.515696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:06.515772 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.515789 (+ 17us) client_negotiation.cc:769] Sending connection context
0621 07:59:06.515866 (+ 77us) client_negotiation.cc:240] Negotiation successful
0621 07:59:06.515950 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":22,"spinlock_wait_cycles":25088,"thread_start_us":114,"threads_started":1}
I20260621 07:59:06.516896 26860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09695f95850a402793a91ec831c0603e" candidate_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba7365107a740daad5244485909d03e" is_pre_election: true
I20260621 07:59:06.516886 26590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09695f95850a402793a91ec831c0603e" candidate_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c31cb0a0134413806b8a7b293c1dba" is_pre_election: true
I20260621 07:59:06.517207 26590 raft_consensus.cc:2454] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a6115eb2d9c45279c9f0995fe423ab8 in term 0.
I20260621 07:59:06.517207 26860 raft_consensus.cc:2454] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a6115eb2d9c45279c9f0995fe423ab8 in term 0.
I20260621 07:59:06.517691 26663 leader_election.cc:302] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6115eb2d9c45279c9f0995fe423ab8, 81c31cb0a0134413806b8a7b293c1dba; no voters:
I20260621 07:59:06.517951 26923 raft_consensus.cc:2781] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:59:06.518039 26923 raft_consensus.cc:482] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:59:06.518100 26923 raft_consensus.cc:3037] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:06.519161 26923 raft_consensus.cc:504] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.519527 26923 leader_election.cc:288] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [CANDIDATE]: Term 1 election: Requested vote from peers 6ba7365107a740daad5244485909d03e (127.11.223.131:34731), 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693)
I20260621 07:59:06.519958 26860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09695f95850a402793a91ec831c0603e" candidate_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba7365107a740daad5244485909d03e"
I20260621 07:59:06.519958 26590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09695f95850a402793a91ec831c0603e" candidate_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c31cb0a0134413806b8a7b293c1dba"
I20260621 07:59:06.520090 26590 raft_consensus.cc:3037] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:06.520090 26860 raft_consensus.cc:3037] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:06.521345 26860 raft_consensus.cc:2454] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a6115eb2d9c45279c9f0995fe423ab8 in term 1.
I20260621 07:59:06.521356 26590 raft_consensus.cc:2454] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a6115eb2d9c45279c9f0995fe423ab8 in term 1.
I20260621 07:59:06.521664 26664 leader_election.cc:302] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6115eb2d9c45279c9f0995fe423ab8, 6ba7365107a740daad5244485909d03e; no voters:
I20260621 07:59:06.521859 26923 raft_consensus.cc:2781] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:06.522114 26923 raft_consensus.cc:686] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 LEADER]: Becoming Leader. State: Replica: 5a6115eb2d9c45279c9f0995fe423ab8, State: Running, Role: LEADER
I20260621 07:59:06.522476 26923 consensus_queue.cc:237] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:06.525628 26447 catalog_manager.cc:5495] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } health_report { overall_health: UNKNOWN } } }
I20260621 07:59:06.547180 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.543988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.11.223.130:35627)
0621 07:59:06.544141 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:06.544145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:06.544157 (+ 12us) server_negotiation.cc:407] Connection header received
0621 07:59:06.544234 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:06.544236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:06.544294 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:06.544433 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:59:06.544652 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.545139 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.545892 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:06.546086 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:06.546158 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:06.546406 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:59:06.546861 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:59:06.546952 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:06.547020 (+ 68us) server_negotiation.cc:299] Negotiation successful
0621 07:59:06.547070 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260621 07:59:06.553036 26590 raft_consensus.cc:1261] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Refusing update from remote peer 5a6115eb2d9c45279c9f0995fe423ab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:06.553049 26860 raft_consensus.cc:1261] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Refusing update from remote peer 5a6115eb2d9c45279c9f0995fe423ab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:06.553738 26927 consensus_queue.cc:1035] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:06.553962 26923 consensus_queue.cc:1035] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:06.559015 26930 mvcc.cc:205] Tried to move back new op lower bound from 7299189745873895424 to 7299189745758789632. Current Snapshot: MvccSnapshot[applied={T|T < 7299189745873895424}]
W20260621 07:59:06.560487 26907 tablet.cc:2369] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:59:06.562253 26931 mvcc.cc:205] Tried to move back new op lower bound from 7299189745873895424 to 7299189745758789632. Current Snapshot: MvccSnapshot[applied={T|T < 7299189745873895424 or (T in {7299189745873895424})}]
I20260621 07:59:06.562990 26932 mvcc.cc:205] Tried to move back new op lower bound from 7299189745873895424 to 7299189745758789632. Current Snapshot: MvccSnapshot[applied={T|T < 7299189745873895424}]
I20260621 07:59:06.569051 26447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:06.569201 26447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:06.571592 26447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Sending DeleteTablet for 3 replicas of tablet 09695f95850a402793a91ec831c0603e
I20260621 07:59:06.572366 26570 tablet_service.cc:1514] Processing DeleteTablet for tablet 09695f95850a402793a91ec831c0603e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:06 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50592
I20260621 07:59:06.572297 26840 tablet_service.cc:1514] Processing DeleteTablet for tablet 09695f95850a402793a91ec831c0603e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:06 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:52404
I20260621 07:59:06.572446 26708 tablet_service.cc:1514] Processing DeleteTablet for tablet 09695f95850a402793a91ec831c0603e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:06 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:56726
I20260621 07:59:06.572875 26942 tablet_replica.cc:330] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: stopping tablet replica
I20260621 07:59:06.572973 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 26486
I20260621 07:59:06.573212 26942 raft_consensus.cc:2229] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:59:06.573756 26942 raft_consensus.cc:2258] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:06.574357 26941 tablet_replica.cc:330] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: stopping tablet replica
I20260621 07:59:06.574734 26941 raft_consensus.cc:2229] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:06.575059 26941 raft_consensus.cc:2258] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:06.575347 26942 ts_tablet_manager.cc:1905] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:06.576630 26941 ts_tablet_manager.cc:1905] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:06.578267 26942 ts_tablet_manager.cc:1918] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:06.578399 26942 log.cc:1196] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/09695f95850a402793a91ec831c0603e
I20260621 07:59:06.578788 26942 ts_tablet_manager.cc:1939] T 09695f95850a402793a91ec831c0603e P 5a6115eb2d9c45279c9f0995fe423ab8: Deleting consensus metadata
I20260621 07:59:06.579998 26941 ts_tablet_manager.cc:1918] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:06.580062 26434 catalog_manager.cc:4841] TS 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627): tablet 09695f95850a402793a91ec831c0603e (table test-table [id=4c5fea5bee084a0cb30d8fb522459f11]) successfully deleted
I20260621 07:59:06.580133 26941 log.cc:1196] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/09695f95850a402793a91ec831c0603e
I20260621 07:59:06.580719 26941 ts_tablet_manager.cc:1939] T 09695f95850a402793a91ec831c0603e P 6ba7365107a740daad5244485909d03e: Deleting consensus metadata
W20260621 07:59:06.581354 26434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:33693 (error 108)
I20260621 07:59:06.581878 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 26640
I20260621 07:59:06.582017 26435 catalog_manager.cc:4841] TS 6ba7365107a740daad5244485909d03e (127.11.223.131:34731): tablet 09695f95850a402793a91ec831c0603e (table test-table [id=4c5fea5bee084a0cb30d8fb522459f11]) successfully deleted
I20260621 07:59:06.583225 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:06.582829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:33693 (local address 127.0.0.1:50596)
0621 07:59:06.583013 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:06.583120 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:33693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260621 07:59:06.583403 26434 catalog_manager.cc:4584] TS 81c31cb0a0134413806b8a7b293c1dba (127.11.223.129:33693): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09695f95850a402793a91ec831c0603e: Network error: Client connection negotiation failed: client connection to 127.11.223.129:33693: connect: Connection refused (error 111)
I20260621 07:59:06.588147 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 26777
I20260621 07:59:06.593497 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 26415
I20260621 07:59:06.599846 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44709
--webserver_interface=127.11.223.190
--webserver_port=40659
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:06.710119 26944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:06.710402 26944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:06.710443 26944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:06.714366 26944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:06.714432 26944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:06.714452 26944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:06.714468 26944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:06.714483 26944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:06.719761 26944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=40659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:06.720922 26944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:06.721964 26944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:06.730291 26952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.730432 26950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.730384 26949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:06.730693 26944 server_base.cc:1034] running on GCE node
I20260621 07:59:06.731312 26944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:06.732532 26944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:06.733752 26944 hybrid_clock.cc:648] HybridClock initialized: now 1782028746733720 us; error 57 us; skew 500 ppm
Jun 21 07:59:06 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:06.736572 26944 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:06.737438 26944 webserver.cc:460] Webserver started at http://127.11.223.190:40659/ using document root <none> and password file <none>
I20260621 07:59:06.738008 26944 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:06.738065 26944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:06.741933 26944 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260621 07:59:06.744077 26959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.745383 26944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 07:59:06.745512 26944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ff2712f0acd6425ca0192b0a11d6d6c6"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "4bbd69095542d333d79e1f06f9db6ca7"
server_key_iv: "fe83e515e0577d7c2eee5f388c41a0ce"
server_key_version: "encryptionkey@0"
I20260621 07:59:06.746063 26944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:06.767021 26944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:06.769995 26944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:06.770198 26944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:06.777421 26944 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:44709
I20260621 07:59:06.777741 27011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:44709 every 8 connection(s)
I20260621 07:59:06.778548 26944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:06.782958 27012 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:59:06.784685 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap starting.
I20260621 07:59:06.786455 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 26944
I20260621 07:59:06.786999 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:33693
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:44709
--webserver_port=36285
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:06.802776 27012 log.cc:824] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:06.814636 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:59:06.815063 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: Bootstrap complete.
I20260621 07:59:06.819173 27012 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:06.819557 27012 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2712f0acd6425ca0192b0a11d6d6c6, State: Initialized, Role: FOLLOWER
I20260621 07:59:06.820129 27012 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:06.820278 27012 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:59:06.820366 27012 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:59:06.820475 27012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 2 FOLLOWER]: Advancing to term 3
I20260621 07:59:06.822477 27012 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:06.822826 27012 leader_election.cc:302] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff2712f0acd6425ca0192b0a11d6d6c6; no voters:
I20260621 07:59:06.823199 27012 leader_election.cc:288] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260621 07:59:06.823288 27016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260621 07:59:06.823480 27016 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [term 3 LEADER]: Becoming Leader. State: Replica: ff2712f0acd6425ca0192b0a11d6d6c6, State: Running, Role: LEADER
I20260621 07:59:06.823837 27016 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } }
I20260621 07:59:06.824510 27012 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:06.825394 27018 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff2712f0acd6425ca0192b0a11d6d6c6. Latest consensus state: current_term: 3 leader_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } } }
I20260621 07:59:06.825512 27018 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:06.825446 27017 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2712f0acd6425ca0192b0a11d6d6c6" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44709 } } }
I20260621 07:59:06.825573 27017 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:06.829416 27031 catalog_manager.cc:1259] Loaded cluster ID: c6de14e62478410fb395f90c2b329ba7
I20260621 07:59:06.829507 27031 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: loading cluster ID for follower catalog manager: success
W20260621 07:59:06.830389 27031 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ff2712f0acd6425ca0192b0a11d6d6c6: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260621 07:59:06.830487 27031 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260621 07:59:06.830669 27025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:06.833050 27025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:06.833477 27025 catalog_manager.cc:1259] Loaded cluster ID: c6de14e62478410fb395f90c2b329ba7
I20260621 07:59:06.833550 27025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260621 07:59:06.834098 27025 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6a3b0eb9ad google::LogMessage::Fail() at ??:0
@ 0x7f6a3b0ee88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6a3b0eb4de google::LogMessage::Flush() at ??:0
@ 0x7f6a3b0eca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6a45a2efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6a45a2fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6a45a2c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6a45a70640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6a3d33d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6a3d33dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6a3d33f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6a3d32e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6a3e9726db start_thread at ??:0
@ 0x7f6a39be961f clone at ??:0
W20260621 07:59:06.945675 27014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:06.946053 27014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:06.946118 27014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:06.952418 27014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:06.952498 27014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:06.952587 27014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:59:06.960448 27014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:33693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=36285
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:06.962216 27014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:06.963776 27014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:06.973340 27053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.973654 27056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:06.973330 27054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:06.973776 27014 server_base.cc:1034] running on GCE node
I20260621 07:59:06.974189 27014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:06.974850 27014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:06.976054 27014 hybrid_clock.cc:648] HybridClock initialized: now 1782028746976025 us; error 50 us; skew 500 ppm
Jun 21 07:59:06 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:06.979012 27014 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:59:06.979919 27014 webserver.cc:460] Webserver started at http://127.11.223.129:36285/ using document root <none> and password file <none>
I20260621 07:59:06.980536 27014 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:06.980599 27014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:06.984639 27014 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:59:06.986856 27063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:06.988533 27014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:59:06.988660 27014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "81c31cb0a0134413806b8a7b293c1dba"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "3f782d168bbb50fe7f053d71d74b22b3"
server_key_iv: "b10befa26ca11d32203fc6cb21e93043"
server_key_version: "encryptionkey@0"
I20260621 07:59:06.989217 27014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:07.009773 27014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:07.013854 27014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:07.014062 27014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:07.014801 27014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:07.016382 27070 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260621 07:59:07.020538 27014 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 07:59:07.020629 27014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260621 07:59:07.020694 27014 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 07:59:07.022960 27014 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 07:59:07.023020 27014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260621 07:59:07.023483 27070 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap starting.
I20260621 07:59:07.034974 27014 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:33693
I20260621 07:59:07.035243 27177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:33693 every 8 connection(s)
I20260621 07:59:07.036157 27014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:59:07.044510 27070 log.cc:824] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:07.044947 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27014
I20260621 07:59:07.045488 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:35627
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:44709
--webserver_port=42651
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:07.052361 27070 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:59:07.052773 27070 tablet_bootstrap.cc:492] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Bootstrap complete.
I20260621 07:59:07.053900 27070 ts_tablet_manager.cc:1397] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Time spent bootstrapping tablet: real 0.031s user 0.025s sys 0.003s
I20260621 07:59:07.056283 27070 raft_consensus.cc:348] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:07.056768 27070 raft_consensus.cc:729] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c31cb0a0134413806b8a7b293c1dba, State: Initialized, Role: FOLLOWER
I20260621 07:59:07.057253 27070 consensus_queue.cc:260] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:07.058199 27070 ts_tablet_manager.cc:1428] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260621 07:59:07.138346 27180 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 07:59:07.038543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:35625)
0621 07:59:07.039132 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.039192 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:07.040485 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:07.137361 (+ 96876us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":146,"threads_started":1}
W20260621 07:59:07.138972 27178 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:44709 (0 consecutive failures): Network error: Failed to ping master at 127.11.223.190:44709: Client connection negotiation failed: client connection to 127.11.223.190:44709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260621 07:59:07.139762 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.139458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:38751)
0621 07:59:07.139606 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.139697 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 07:59:07.140514 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.140246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:60095)
0621 07:59:07.140401 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.140447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 07:59:07.140779 27178 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 07:59:07.188118 27182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:07.188475 27182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:07.188581 27182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:07.192659 27182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:07.192740 27182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:07.192850 27182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:59:07.198203 27182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:35627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=42651
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:07.199316 27182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:07.200387 27182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:07.209640 27191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:07.209702 27189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:07.209822 27182 server_base.cc:1034] running on GCE node
W20260621 07:59:07.209671 27188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:07.210346 27182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:07.210942 27182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:07.212105 27182 hybrid_clock.cc:648] HybridClock initialized: now 1782028747212072 us; error 45 us; skew 500 ppm
Jun 21 07:59:07 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028747, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:07.214913 27182 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:59:07.215760 27182 webserver.cc:460] Webserver started at http://127.11.223.130:42651/ using document root <none> and password file <none>
I20260621 07:59:07.216395 27182 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:07.216485 27182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:07.220141 27182 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:59:07.222230 27198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:07.223423 27182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:07.223555 27182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "5a6115eb2d9c45279c9f0995fe423ab8"
format_stamp: "Formatted at 2026-06-21 07:59:04 on dist-test-slave-0pn3"
server_key: "415bfaa5591d9b33b8e106c7857c7972"
server_key_iv: "3ebf5fcbbb882035c9a3b6be2c362744"
server_key_version: "encryptionkey@0"
I20260621 07:59:07.224133 27182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:07.243014 27182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:07.246580 27182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:07.246806 27182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:07.247462 27182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:07.248338 27182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:07.248407 27182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:07.248502 27182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:07.248538 27182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:07.257444 27182 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:35627
I20260621 07:59:07.257468 27311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:35627 every 8 connection(s)
I20260621 07:59:07.258503 27182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:59:07.262274 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.260686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:44365)
0621 07:59:07.261151 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.261595 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":103,"threads_started":1}
W20260621 07:59:07.262717 27312 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:44709 (0 consecutive failures): Network error: Failed to ping master at 127.11.223.190:44709: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
I20260621 07:59:07.263304 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.263070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:36477)
0621 07:59:07.263191 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.263251 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260621 07:59:07.263900 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.263701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:46845)
0621 07:59:07.263813 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.263852 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260621 07:59:07.264111 27312 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260621 07:59:07.264500 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27182
I20260621 07:59:07.264854 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:34731
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:44709
--webserver_port=41531
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:07.375564 27315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:07.375865 27315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:07.375941 27315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:07.379952 27315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:07.380053 27315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:07.380143 27315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:59:07.385389 27315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:34731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=41531
--tserver_master_addrs=127.11.223.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:07.386476 27315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:07.387553 27315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:07.397646 27323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:07.397692 27321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:07.397710 27315 server_base.cc:1034] running on GCE node
W20260621 07:59:07.397720 27320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:07.398387 27315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:07.399021 27315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:07.400180 27315 hybrid_clock.cc:648] HybridClock initialized: now 1782028747400149 us; error 45 us; skew 500 ppm
Jun 21 07:59:07 dist-test-slave-0pn3 krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028747, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:07.402873 27315 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:59:07.403748 27315 webserver.cc:460] Webserver started at http://127.11.223.131:41531/ using document root <none> and password file <none>
I20260621 07:59:07.404352 27315 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:07.404453 27315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:07.408402 27315 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 07:59:07.410467 27330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:07.411830 27315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 07:59:07.411983 27315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "6ba7365107a740daad5244485909d03e"
format_stamp: "Formatted at 2026-06-21 07:59:05 on dist-test-slave-0pn3"
server_key: "09cedb377ffc53f8faf0a39a97622087"
server_key_iv: "88d18a47d0da60e5c014c121660b299d"
server_key_version: "encryptionkey@0"
I20260621 07:59:07.412596 27315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:07.422835 27315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:07.425869 27315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:07.426074 27315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:07.426678 27315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:07.427616 27315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:07.427687 27315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:07.427759 27315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:07.427819 27315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:07.437081 27315 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:34731
I20260621 07:59:07.437103 27443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:34731 every 8 connection(s)
I20260621 07:59:07.438539 27315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:59:07.441270 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27315
I20260621 07:59:07.442239 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.440772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:54697)
0621 07:59:07.441218 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.441355 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":151,"threads_started":1}
W20260621 07:59:07.442747 27444 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:44709 (0 consecutive failures): Network error: Failed to ping master at 127.11.223.190:44709: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
I20260621 07:59:07.443267 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.443072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:45273)
0621 07:59:07.443174 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.443219 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260621 07:59:07.443802 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:07.443628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:51039)
0621 07:59:07.443717 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:07.443756 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260621 07:59:07.443958 27444 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260621 07:59:08.141767 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.141371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:48315)
0621 07:59:08.141610 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:08.141661 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260621 07:59:08.265086 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.264712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:55757)
0621 07:59:08.264938 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:08.264989 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260621 07:59:08.275904 27450 raft_consensus.cc:482] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:08.276136 27450 raft_consensus.cc:504] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:08.277307 27450 leader_election.cc:288] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ba7365107a740daad5244485909d03e (127.11.223.131:34731), 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
Jun 21 07:59:08 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.130@KRBTEST.COM
Jun 21 07:59:08 dist-test-slave-0pn3 krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028746, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.131@KRBTEST.COM
I20260621 07:59:08.295956 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.277753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52169 (local address 127.11.223.130:35627)
0621 07:59:08.278150 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:08.278169 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:08.278210 (+ 41us) server_negotiation.cc:407] Connection header received
0621 07:59:08.279125 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:08.279131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:08.279600 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:08.279974 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:08.282000 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.283597 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.285674 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.286401 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.286572 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:08.290245 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:08.290254 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:08.290273 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:08.290314 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:08.292663 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293335 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:08.293340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:08.293345 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:08.293493 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293913 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:08.293915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:08.293916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:08.294525 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:08.294644 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:08.295471 (+ 827us) server_negotiation.cc:299] Negotiation successful
0621 07:59:08.295765 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":80,"threads_started":1}
I20260621 07:59:08.295953 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.277753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:44909 (local address 127.11.223.131:34731)
0621 07:59:08.278150 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:08.278167 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:08.278210 (+ 43us) server_negotiation.cc:407] Connection header received
0621 07:59:08.279112 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:08.279121 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:08.279600 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:08.279973 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:08.282000 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.283597 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.285672 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.286401 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.286572 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:08.290573 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:08.290579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:08.290590 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:08.290621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:08.292662 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293541 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:08.293548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:08.293556 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:08.293697 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293994 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:08.293997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:08.293998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:08.294509 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:08.294644 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:08.295471 (+ 827us) server_negotiation.cc:299] Negotiation successful
0621 07:59:08.295765 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":75,"threads_started":1}
I20260621 07:59:08.296289 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.277579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:34731 (local address 127.11.223.129:44909)
0621 07:59:08.277921 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:08.277937 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:08.278046 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:08.280118 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:08.280125 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:08.280818 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:08.281757 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:08.281796 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.283804 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.283808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:08.285530 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:08.285543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.286581 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.286585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:08.286922 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:08.287712 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:08.287741 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:08.290349 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:08.293254 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293256 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:08.293258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:08.293424 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:08.293780 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:08.293785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:08.293875 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:08.294703 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:08.294710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:08.295077 (+ 367us) client_negotiation.cc:769] Sending connection context
0621 07:59:08.295300 (+ 223us) client_negotiation.cc:240] Negotiation successful
0621 07:59:08.295760 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":444,"spinlock_wait_cycles":80896}
I20260621 07:59:08.297013 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.277579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35627 (local address 127.11.223.129:52169)
0621 07:59:08.277921 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:08.277937 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:08.278046 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:08.280118 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:08.280125 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:08.280818 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:08.281757 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:08.281789 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.283804 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.283807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:08.285534 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:08.285543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:08.286581 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:08.286585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:08.286923 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:08.287712 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:08.287741 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:08.289928 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:08.292837 (+ 2909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:08.292844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:08.292864 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:08.293167 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:08.293603 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:08.293605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:08.293607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:08.293807 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:08.294703 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:08.294710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:08.295077 (+ 367us) client_negotiation.cc:769] Sending connection context
0621 07:59:08.295300 (+ 223us) client_negotiation.cc:240] Negotiation successful
0621 07:59:08.296891 (+ 1591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":1791,"spinlock_wait_cycles":14208,"thread_start_us":95,"threads_started":1}
W20260621 07:59:08.299149 27067 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ba7365107a740daad5244485909d03e (127.11.223.131:34731): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
W20260621 07:59:08.299328 27066 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
I20260621 07:59:08.299427 27066 leader_election.cc:302] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81c31cb0a0134413806b8a7b293c1dba; no voters: 5a6115eb2d9c45279c9f0995fe423ab8, 6ba7365107a740daad5244485909d03e
I20260621 07:59:08.299711 27450 raft_consensus.cc:2726] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:59:08.444993 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:08.444578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:35681)
0621 07:59:08.444825 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:08.444885 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260621 07:59:09.142876 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:09.142471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:58465)
0621 07:59:09.142717 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:09.142773 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260621 07:59:09.266150 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:09.265766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:40505)
0621 07:59:09.266008 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:09.266066 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260621 07:59:09.446147 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:09.445724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:35289)
0621 07:59:09.445992 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:09.446043 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":73,"threads_started":1}
I20260621 07:59:10.144212 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:10.143745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:47533)
0621 07:59:10.144050 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:10.144114 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":85,"threads_started":1}
I20260621 07:59:10.182725 27459 raft_consensus.cc:482] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:10.182858 27459 raft_consensus.cc:504] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:10.183252 27459 leader_election.cc:288] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ba7365107a740daad5244485909d03e (127.11.223.131:34731), 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
W20260621 07:59:10.184041 27067 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ba7365107a740daad5244485909d03e (127.11.223.131:34731): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
W20260621 07:59:10.184159 27066 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
I20260621 07:59:10.184267 27066 leader_election.cc:302] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81c31cb0a0134413806b8a7b293c1dba; no voters: 5a6115eb2d9c45279c9f0995fe423ab8, 6ba7365107a740daad5244485909d03e
I20260621 07:59:10.184449 27459 raft_consensus.cc:2726] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:59:10.267381 27460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:10.266901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:51943)
0621 07:59:10.267178 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:10.267245 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260621 07:59:10.447361 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:10.446960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:45223)
0621 07:59:10.447215 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:10.447272 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260621 07:59:11.145371 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:11.144966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:48481)
0621 07:59:11.145219 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:11.145274 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260621 07:59:11.268610 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:11.268154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:36179)
0621 07:59:11.268456 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:11.268516 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
I20260621 07:59:11.448611 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:11.448116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:35077)
0621 07:59:11.448437 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:11.448499 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260621 07:59:12.146994 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:12.146444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:33887)
0621 07:59:12.146794 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:12.146867 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":78,"threads_started":1}
I20260621 07:59:12.269793 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:12.269414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:50511)
0621 07:59:12.269645 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:12.269700 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260621 07:59:12.449867 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:12.449418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:40101)
0621 07:59:12.449699 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:12.449766 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260621 07:59:13.118062 27468 raft_consensus.cc:482] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:13.118194 27468 raft_consensus.cc:504] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:13.118604 27468 leader_election.cc:288] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ba7365107a740daad5244485909d03e (127.11.223.131:34731), 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
W20260621 07:59:13.119439 27067 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ba7365107a740daad5244485909d03e (127.11.223.131:34731): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
W20260621 07:59:13.119542 27066 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
I20260621 07:59:13.119599 27066 leader_election.cc:302] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81c31cb0a0134413806b8a7b293c1dba; no voters: 5a6115eb2d9c45279c9f0995fe423ab8, 6ba7365107a740daad5244485909d03e
I20260621 07:59:13.119995 27468 raft_consensus.cc:2726] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:59:13.148203 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:13.147774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:53689)
0621 07:59:13.148045 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:13.148104 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":101,"threads_started":1}
I20260621 07:59:13.270988 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:13.270579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:53893)
0621 07:59:13.270827 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:13.270887 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260621 07:59:13.451188 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:13.450728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:45627)
0621 07:59:13.451021 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:13.451087 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
I20260621 07:59:14.149441 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:14.149005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:38507)
0621 07:59:14.149273 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:14.149334 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260621 07:59:14.272186 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:14.271762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:59747)
0621 07:59:14.272043 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:14.272094 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260621 07:59:14.452415 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:14.451906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:59983)
0621 07:59:14.452202 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:14.452265 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":74,"threads_started":1}
I20260621 07:59:15.150857 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:15.150398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:52497)
0621 07:59:15.150695 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:15.150759 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
I20260621 07:59:15.273463 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:15.273030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:37781)
0621 07:59:15.273288 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:15.273353 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260621 07:59:15.453624 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:15.453074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:60293)
0621 07:59:15.453469 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:15.453527 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":91,"threads_started":1}
I20260621 07:59:16.152132 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:16.151700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:40401)
0621 07:59:16.151972 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:16.152033 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
I20260621 07:59:16.274583 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:16.274189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:42839)
0621 07:59:16.274434 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:16.274488 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260621 07:59:16.454939 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:16.454488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:38727)
0621 07:59:16.454765 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:16.454841 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
I20260621 07:59:17.153651 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:17.153111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:39151)
0621 07:59:17.153431 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:17.153528 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":105,"threads_started":1}
I20260621 07:59:17.276021 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:17.275591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:45689)
0621 07:59:17.275849 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:17.275924 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260621 07:59:17.456480 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:17.455938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:48533)
0621 07:59:17.456275 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:17.456361 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":120,"threads_started":1}
I20260621 07:59:17.736699 27484 raft_consensus.cc:482] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:17.736840 27484 raft_consensus.cc:504] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7365107a740daad5244485909d03e" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 34731 } } peers { permanent_uuid: "5a6115eb2d9c45279c9f0995fe423ab8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35627 } } peers { permanent_uuid: "81c31cb0a0134413806b8a7b293c1dba" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 33693 } }
I20260621 07:59:17.737313 27484 leader_election.cc:288] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ba7365107a740daad5244485909d03e (127.11.223.131:34731), 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627)
W20260621 07:59:17.738170 27067 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ba7365107a740daad5244485909d03e (127.11.223.131:34731): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
W20260621 07:59:17.738279 27066 leader_election.cc:341] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a6115eb2d9c45279c9f0995fe423ab8 (127.11.223.130:35627): Not found: Tablet not found: 09695f95850a402793a91ec831c0603e
I20260621 07:59:17.738408 27066 leader_election.cc:302] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81c31cb0a0134413806b8a7b293c1dba; no voters: 5a6115eb2d9c45279c9f0995fe423ab8, 6ba7365107a740daad5244485909d03e
I20260621 07:59:17.738564 27484 raft_consensus.cc:2726] T 09695f95850a402793a91ec831c0603e P 81c31cb0a0134413806b8a7b293c1dba [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260621 07:59:18.154968 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:18.154542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:36727)
0621 07:59:18.154808 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:18.154867 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260621 07:59:18.277302 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:18.276859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:33019)
0621 07:59:18.277136 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:18.277202 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260621 07:59:18.457890 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:18.457456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:47333)
0621 07:59:18.457705 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:18.457763 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260621 07:59:19.156133 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:19.155697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:59703)
0621 07:59:19.155982 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:19.156039 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
I20260621 07:59:19.278524 27489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:19.278101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:44535)
0621 07:59:19.278360 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:19.278415 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260621 07:59:19.459282 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:19.458737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:37953)
0621 07:59:19.459083 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:19.459165 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":86,"threads_started":1}
I20260621 07:59:20.157466 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:20.157040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:47555)
0621 07:59:20.157313 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:20.157373 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260621 07:59:20.279868 27492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:20.279423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:46583)
0621 07:59:20.279713 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:20.279774 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260621 07:59:20.460558 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:20.460125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:33143)
0621 07:59:20.460410 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:20.460463 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260621 07:59:21.158840 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:21.158371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:51615)
0621 07:59:21.158654 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:21.158721 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260621 07:59:21.281104 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:21.280690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:41277)
0621 07:59:21.280950 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:21.281009 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260621 07:59:21.461838 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:21.461386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:39581)
0621 07:59:21.461672 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:21.461734 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
I20260621 07:59:22.160298 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:22.159898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.129:56451)
0621 07:59:22.160147 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:22.160203 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
I20260621 07:59:22.282523 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:22.282088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.130:57473)
0621 07:59:22.282361 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:22.282421 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260621 07:59:22.451817 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27014
I20260621 07:59:22.457489 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27182
I20260621 07:59:22.462919 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27315
I20260621 07:59:22.463218 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:22.462771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44709 (local address 127.11.223.131:38563)
0621 07:59:22.463013 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:22.463091 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:44709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
2026-06-21T07:59:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19257 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 07:59:22 dist-test-slave-0pn3 krb5kdc[27502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 07:59:22 dist-test-slave-0pn3 krb5kdc[27502](info): set up 2 sockets
Jun 21 07:59:22 dist-test-slave-0pn3 krb5kdc[27502](info): commencing operation
krb5kdc: starting...
W20260621 07:59:23.488759 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 07:59:23 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T07:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T07:59:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 07:59:23.608675 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32927
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:23.720306 27515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:23.720674 27515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:23.720757 27515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:23.724844 27515 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260621 07:59:23.724936 27515 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260621 07:59:23.724958 27515 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260621 07:59:23.724980 27515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:23.725023 27515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:23.725057 27515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:23.725111 27515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:23.725149 27515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:23.730548 27515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:23.731724 27515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:23.732841 27515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:23.741636 27520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:23.741744 27515 server_base.cc:1034] running on GCE node
W20260621 07:59:23.741662 27521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:23.741636 27523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:23.742463 27515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:23.743566 27515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:23.744724 27515 hybrid_clock.cc:648] HybridClock initialized: now 1782028763744700 us; error 41 us; skew 500 ppm
Jun 21 07:59:23 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028763, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:23.752472 27515 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:23.753391 27515 webserver.cc:460] Webserver started at http://127.11.223.190:41235/ using document root <none> and password file <none>
I20260621 07:59:23.753937 27515 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:23.753993 27515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:23.754159 27515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:23.756271 27515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "681ad4cedcfa42da9f31c0e950cc2369"
format_stamp: "Formatted at 2026-06-21 07:59:23 on dist-test-slave-0pn3"
server_key: "00844342186bd83f991da4c0c2198c35"
server_key_iv: "006f069e23b5f7451231bba04fb42c6d"
server_key_version: "encryptionkey@0"
I20260621 07:59:23.756886 27515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "681ad4cedcfa42da9f31c0e950cc2369"
format_stamp: "Formatted at 2026-06-21 07:59:23 on dist-test-slave-0pn3"
server_key: "00844342186bd83f991da4c0c2198c35"
server_key_iv: "006f069e23b5f7451231bba04fb42c6d"
server_key_version: "encryptionkey@0"
I20260621 07:59:23.760699 27515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260621 07:59:23.763101 27530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:23.764503 27515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 07:59:23.764632 27515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "681ad4cedcfa42da9f31c0e950cc2369"
format_stamp: "Formatted at 2026-06-21 07:59:23 on dist-test-slave-0pn3"
server_key: "00844342186bd83f991da4c0c2198c35"
server_key_iv: "006f069e23b5f7451231bba04fb42c6d"
server_key_version: "encryptionkey@0"
I20260621 07:59:23.764744 27515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:23.781337 27515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:23.784621 27515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:23.784854 27515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:23.792148 27515 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:32927
I20260621 07:59:23.792482 27582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:32927 every 8 connection(s)
I20260621 07:59:23.793207 27515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:23.795393 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27515
I20260621 07:59:23.795506 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 07:59:23.795831 12158 external_mini_cluster.cc:1428] Setting key 2aae69683241f215b3378eeae833a61f
I20260621 07:59:23.796236 27583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:23.803293 27583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap starting.
Jun 21 07:59:23 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:23.805800 27583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:23.806553 27583 log.cc:824] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:23.808531 27583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: No bootstrap required, opened a new log
I20260621 07:59:23.810460 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:23.797258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.11.223.190:32927)
0621 07:59:23.797672 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:23.797694 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:23.797730 (+ 36us) server_negotiation.cc:407] Connection header received
0621 07:59:23.798387 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:23.798402 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:23.798723 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:23.798992 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:23.799895 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:23.800954 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:23.802143 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:23.802592 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:23.802710 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:23.805426 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:23.805431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:23.805445 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:23.805472 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:23.807568 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:23.808026 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:23.808031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:23.808037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:23.808162 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:23.808477 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:23.808481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:23.808483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:23.808927 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:23.809142 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:23.809436 (+ 294us) server_negotiation.cc:299] Negotiation successful
0621 07:59:23.809758 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":159,"threads_started":1}
I20260621 07:59:23.812124 27583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:23.812393 27583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:23.812479 27583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681ad4cedcfa42da9f31c0e950cc2369, State: Initialized, Role: FOLLOWER
I20260621 07:59:23.812985 27583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:23.813120 27583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:59:23.813162 27583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:59:23.813246 27583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:23.814154 27583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:23.814443 27583 leader_election.cc:302] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 681ad4cedcfa42da9f31c0e950cc2369; no voters:
I20260621 07:59:23.814692 27583 leader_election.cc:288] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 07:59:23.814842 27588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:23.815199 27588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 LEADER]: Becoming Leader. State: Replica: 681ad4cedcfa42da9f31c0e950cc2369, State: Running, Role: LEADER
I20260621 07:59:23.815521 27588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:23.815946 27583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:23.817330 27589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "681ad4cedcfa42da9f31c0e950cc2369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } } }
I20260621 07:59:23.817401 27590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 681ad4cedcfa42da9f31c0e950cc2369. Latest consensus state: current_term: 1 leader_uuid: "681ad4cedcfa42da9f31c0e950cc2369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } } }
I20260621 07:59:23.817483 27589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:23.817487 27590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:23.818078 27600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:23.821609 27600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:23.826841 27600 catalog_manager.cc:1347] Generated new cluster ID: 5357fa4dfebb4cc490ff29d74c0a6cbe
I20260621 07:59:23.826917 27600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:59:23.837119 27600 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 07:59:23.838361 27600 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:59:23.845777 27600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Generated new TSK 0
I20260621 07:59:23.846640 27600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 07:59:23.876771 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:32927
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:23.988077 27609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:23.988351 27609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:23.988395 27609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:23.992632 27609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:23.992693 27609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:23.992770 27609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:59:23.998349 27609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:23.999388 27609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:24.000461 27609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:24.010281 27615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:24.010286 27617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:24.010324 27609 server_base.cc:1034] running on GCE node
W20260621 07:59:24.010286 27614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:24.010886 27609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:24.011555 27609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:24.012740 27609 hybrid_clock.cc:648] HybridClock initialized: now 1782028764012727 us; error 29 us; skew 500 ppm
Jun 21 07:59:24 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028764, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:24.015482 27609 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:59:24.016415 27609 webserver.cc:460] Webserver started at http://127.11.223.129:42083/ using document root <none> and password file <none>
I20260621 07:59:24.017041 27609 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:24.017104 27609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:24.017272 27609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:24.019384 27609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "2a8aa6a16e5b44069db84e1db26d4832"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "a71a3f8063914081bee34ab0408a1638"
server_key_iv: "78996aef8a9629b6cea655e11d8ce1f9"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.019914 27609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "2a8aa6a16e5b44069db84e1db26d4832"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "a71a3f8063914081bee34ab0408a1638"
server_key_iv: "78996aef8a9629b6cea655e11d8ce1f9"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.023612 27609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260621 07:59:24.026036 27624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.027302 27609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:24.027426 27609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "2a8aa6a16e5b44069db84e1db26d4832"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "a71a3f8063914081bee34ab0408a1638"
server_key_iv: "78996aef8a9629b6cea655e11d8ce1f9"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.027536 27609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:24.039530 27609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:24.042477 27609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:24.042688 27609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:24.043314 27609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:24.044167 27609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:24.044215 27609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.044289 27609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:24.044395 27609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.053284 27609 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:36015
I20260621 07:59:24.053309 27737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:36015 every 8 connection(s)
I20260621 07:59:24.054208 27609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:59:24.063148 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27609
I20260621 07:59:24.063297 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
Jun 21 07:59:24 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028764, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:24.063653 12158 external_mini_cluster.cc:1428] Setting key 8d3015aa49bb6aab94c9609a6aa03c12
I20260621 07:59:24.068410 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.056130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:53683 (local address 127.11.223.190:32927)
0621 07:59:24.056278 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.056283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.057438 (+ 1155us) server_negotiation.cc:407] Connection header received
0621 07:59:24.058254 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.058256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.058303 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.058379 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:24.059519 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.060062 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.061222 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.061451 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.061532 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.064262 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:24.064266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:24.064269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:24.064299 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:24.065910 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.066584 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.066587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.066588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.066660 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.067061 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.067066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.067078 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.067343 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:24.067452 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.068018 (+ 566us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.068182 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 07:59:24.069245 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.056592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:53683)
0621 07:59:24.057273 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.057315 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.058076 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.058545 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.058551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.058915 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:24.059390 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.059403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.060172 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.060174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.061107 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.061117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.061584 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.061587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.061731 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.062295 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:24.062310 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:24.064071 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:24.066032 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:24.066041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:24.066061 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:24.066458 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:24.066749 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:24.066756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:24.066760 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:24.066959 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:24.067456 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:24.067461 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:24.067714 (+ 253us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.067949 (+ 235us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.068215 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":115,"threads_started":1}
I20260621 07:59:24.070598 27738 heartbeater.cc:347] Connected to a master server at 127.11.223.190:32927
I20260621 07:59:24.070861 27738 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:24.071507 27738 heartbeater.cc:511] Master 127.11.223.190:32927 requested a full tablet report, sending...
I20260621 07:59:24.073208 27547 ts_manager.cc:194] Registered new tserver with Master: 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015)
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 07:59:24.074440 27547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:53683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 07:59:24.091212 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:32927
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:24.204437 27743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:24.204718 27743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:24.204762 27743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:24.209022 27743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:24.209088 27743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:24.209187 27743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:59:24.214483 27743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:24.215574 27743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:24.216708 27743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:24.226677 27749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:24.226846 27743 server_base.cc:1034] running on GCE node
W20260621 07:59:24.226728 27748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:24.226717 27751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:24.227406 27743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:24.228065 27743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:24.229261 27743 hybrid_clock.cc:648] HybridClock initialized: now 1782028764229235 us; error 39 us; skew 500 ppm
Jun 21 07:59:24 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028764, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:24.232157 27743 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:59:24.233096 27743 webserver.cc:460] Webserver started at http://127.11.223.130:40585/ using document root <none> and password file <none>
I20260621 07:59:24.233692 27743 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:24.233785 27743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:24.234001 27743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:24.236039 27743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "4965d9216c0d426182adbd09fd3c8e68"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "2a2a6a97e19a5bc6e345f112cca35703"
server_key_iv: "feacebc4dd952c72f052627e1c2a97a4"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.236629 27743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "4965d9216c0d426182adbd09fd3c8e68"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "2a2a6a97e19a5bc6e345f112cca35703"
server_key_iv: "feacebc4dd952c72f052627e1c2a97a4"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.240200 27743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:59:24.242534 27758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.243728 27743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:24.243877 27743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "4965d9216c0d426182adbd09fd3c8e68"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "2a2a6a97e19a5bc6e345f112cca35703"
server_key_iv: "feacebc4dd952c72f052627e1c2a97a4"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.244002 27743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:24.277444 27743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:24.280419 27743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:24.280624 27743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:24.281251 27743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:24.282094 27743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:24.282163 27743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.282234 27743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:24.282276 27743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.291369 27743 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:45779
I20260621 07:59:24.291389 27871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:45779 every 8 connection(s)
I20260621 07:59:24.292389 27743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:59:24.298182 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27743
I20260621 07:59:24.298321 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 07:59:24.298611 12158 external_mini_cluster.cc:1428] Setting key 000040bdcbb071ecc96fdb38e6897d29
Jun 21 07:59:24 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028764, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:24.307212 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.294193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:55607 (local address 127.11.223.190:32927)
0621 07:59:24.294350 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.294355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.295722 (+ 1367us) server_negotiation.cc:407] Connection header received
0621 07:59:24.296633 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.296636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.296684 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.296755 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:24.297849 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.298661 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.300045 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.300277 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.300460 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.303170 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:24.303176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:24.303182 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:24.303219 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:24.304971 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.305510 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.305515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.305519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.305624 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.305955 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.305960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.305962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.306250 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:24.306370 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.306862 (+ 492us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.307008 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:59:24.308125 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.294649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:55607)
0621 07:59:24.295556 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.295600 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.296432 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.296891 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.296897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.297242 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:24.297724 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.297737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.298803 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.298806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.299898 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.299911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.300499 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.300503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.300682 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.301306 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:24.301321 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:24.302995 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:24.305112 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:24.305118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:24.305140 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:24.305406 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:24.305730 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:24.305732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:24.305733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:24.305864 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:24.306354 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:24.306359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:24.306617 (+ 258us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.306780 (+ 163us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.307055 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":540,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 07:59:24.309382 27872 heartbeater.cc:347] Connected to a master server at 127.11.223.190:32927
I20260621 07:59:24.309640 27872 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:24.310148 27872 heartbeater.cc:511] Master 127.11.223.190:32927 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 07:59:24.311316 27547 ts_manager.cc:194] Registered new tserver with Master: 4965d9216c0d426182adbd09fd3c8e68 (127.11.223.130:45779)
I20260621 07:59:24.311926 27547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:55607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 07:59:24.327554 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:32927
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:24.439567 27877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:24.439827 27877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:24.439869 27877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:24.444032 27877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:24.444095 27877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:24.444182 27877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:59:24.449821 27877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:24.450922 27877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:24.452016 27877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:24.461668 27882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:24.461663 27883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:24.461668 27877 server_base.cc:1034] running on GCE node
W20260621 07:59:24.461696 27885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:24.462396 27877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:24.463061 27877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:24.464288 27877 hybrid_clock.cc:648] HybridClock initialized: now 1782028764464273 us; error 40 us; skew 500 ppm
Jun 21 07:59:24 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028764, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:24.467031 27877 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:59:24.467945 27877 webserver.cc:460] Webserver started at http://127.11.223.131:44429/ using document root <none> and password file <none>
I20260621 07:59:24.468654 27877 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:24.468745 27877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:24.468977 27877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 07:59:24.471016 27877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "8bac26543b0e4662a779fcd33e0c376b"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "dc8c439ad71520b870a87d88c2615451"
server_key_iv: "34e96fe13ba51d6ab2771347c19e0525"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.471580 27877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "8bac26543b0e4662a779fcd33e0c376b"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "dc8c439ad71520b870a87d88c2615451"
server_key_iv: "34e96fe13ba51d6ab2771347c19e0525"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.475421 27877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260621 07:59:24.477854 27892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.479022 27877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 07:59:24.479182 27877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "8bac26543b0e4662a779fcd33e0c376b"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "dc8c439ad71520b870a87d88c2615451"
server_key_iv: "34e96fe13ba51d6ab2771347c19e0525"
server_key_version: "encryptionkey@0"
I20260621 07:59:24.479297 27877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:24.501245 27877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:24.504117 27877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:24.504359 27877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:24.504974 27877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:24.505800 27877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:24.505870 27877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.505939 27877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:24.505985 27877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:24.514691 27877 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:35971
I20260621 07:59:24.514720 28005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:35971 every 8 connection(s)
I20260621 07:59:24.515667 27877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:59:24.524575 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 27877
I20260621 07:59:24.524737 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
Jun 21 07:59:24 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028764, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:24.525070 12158 external_mini_cluster.cc:1428] Setting key f6a669b0fd3f0a925a8257a2e84b7e7b
I20260621 07:59:24.529346 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.517628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:47305 (local address 127.11.223.190:32927)
0621 07:59:24.517785 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.517790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.518836 (+ 1046us) server_negotiation.cc:407] Connection header received
0621 07:59:24.519679 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.519681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.519728 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.519801 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:24.521028 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.521542 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.522637 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.522851 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.522930 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.525787 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:24.525791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:24.525793 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:24.525818 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:24.527468 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.527955 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.527958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.527959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.528031 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.528370 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.528373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.528378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.528580 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:24.528659 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.529128 (+ 469us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.529228 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:59:24.530231 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.518045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:47305)
0621 07:59:24.518689 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.518732 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.519495 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.519973 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.519980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.520368 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:24.520884 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.520898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.521656 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.521658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.522517 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.522526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.522982 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.522984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.523142 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.523757 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:24.523775 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:24.525564 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:24.527574 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:24.527580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:24.527603 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:24.527870 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:24.528109 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:24.528111 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:24.528112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:24.528253 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:24.528671 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:24.528676 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:24.528921 (+ 245us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.529083 (+ 162us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.529357 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":106,"threads_started":1}
I20260621 07:59:24.531387 28006 heartbeater.cc:347] Connected to a master server at 127.11.223.190:32927
I20260621 07:59:24.531634 28006 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:24.532155 28006 heartbeater.cc:511] Master 127.11.223.190:32927 requested a full tablet report, sending...
I20260621 07:59:24.533255 27547 ts_manager.cc:194] Registered new tserver with Master: 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971)
I20260621 07:59:24.533823 27547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:47305
I20260621 07:59:24.539247 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:59:24.549315 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.541265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.11.223.190:32927)
0621 07:59:24.541427 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.541432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.541615 (+ 183us) server_negotiation.cc:407] Connection header received
0621 07:59:24.541788 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.541795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.541835 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.541917 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:24.542635 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.543201 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.544073 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.544289 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.544448 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.545913 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:24.545917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:24.545920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:24.545946 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:24.547411 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.547911 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.547915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.547917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.548025 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:24.548303 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:24.548307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:24.548309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:24.548576 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:24.548725 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.548990 (+ 265us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.549130 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 07:59:24.552117 27547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:59:24.555308 27547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:59:24.570278 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.564593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.11.223.131:35971)
0621 07:59:24.565731 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.565740 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.565758 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:59:24.565819 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.565822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.565965 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.566376 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:24.566507 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.567355 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.568908 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.569570 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.569752 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.569888 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.570019 (+ 131us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.570115 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":96,"threads_started":1}
I20260621 07:59:24.570266 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.564428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:35971 (local address 127.0.0.1:39404)
0621 07:59:24.565094 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.565108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.565216 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.566131 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.566133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.566146 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:24.566238 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.566247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.567515 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.567517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.568760 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.568766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.569739 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.569742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.569846 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.569882 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.569993 (+ 111us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.570135 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":41,"thread_start_us":102,"threads_started":1}
I20260621 07:59:24.570559 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.564907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46114 (local address 127.11.223.129:36015)
0621 07:59:24.565347 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.565362 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.565602 (+ 240us) server_negotiation.cc:407] Connection header received
0621 07:59:24.565696 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.565701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.565890 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.566039 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:24.566288 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.567316 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.568886 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.569684 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.569813 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.569946 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.570338 (+ 392us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.570423 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":128,"threads_started":1}
I20260621 07:59:24.570562 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.563922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.11.223.130:45779)
0621 07:59:24.564707 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.564720 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.564746 (+ 26us) server_negotiation.cc:407] Connection header received
0621 07:59:24.564820 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.564825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.565052 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.565593 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:24.565787 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.566962 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.568628 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.569555 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.570145 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.570280 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.570332 (+ 52us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.570420 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":161,"threads_started":1}
I20260621 07:59:24.570715 28017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.563762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:45779 (local address 127.0.0.1:50332)
0621 07:59:24.564144 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.564202 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.564374 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.565234 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.565244 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.565273 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:24.565425 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.565432 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.567108 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.567111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.568502 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.568512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.569871 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.569874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.569965 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.569991 (+ 26us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.570040 (+ 49us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.570598 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":473,"thread_start_us":92,"threads_started":1}
I20260621 07:59:24.571177 28019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.564541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36015 (local address 127.0.0.1:46114)
0621 07:59:24.565398 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.565413 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.565499 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.566044 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.566050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.566072 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:24.566180 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.566186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.567439 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.567444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.568756 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.568766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.570001 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.570004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.570096 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.570117 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.570176 (+ 59us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.570244 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":11,"thread_start_us":68,"threads_started":1}
I20260621 07:59:24.573863 27672 tablet_service.cc:1467] Processing CreateTablet for tablet ff36a869943d40aa9a7733b7d6181746 (DEFAULT_TABLE table=test-table [id=a7385f15faf648c99e7cb805fe521a50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:24.573882 27940 tablet_service.cc:1467] Processing CreateTablet for tablet ff36a869943d40aa9a7733b7d6181746 (DEFAULT_TABLE table=test-table [id=a7385f15faf648c99e7cb805fe521a50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:24.573882 27806 tablet_service.cc:1467] Processing CreateTablet for tablet ff36a869943d40aa9a7733b7d6181746 (DEFAULT_TABLE table=test-table [id=a7385f15faf648c99e7cb805fe521a50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:24.574911 27806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff36a869943d40aa9a7733b7d6181746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:24.574916 27940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff36a869943d40aa9a7733b7d6181746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:24.574911 27672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff36a869943d40aa9a7733b7d6181746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:24.581079 28023 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap starting.
I20260621 07:59:24.582230 28024 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Bootstrap starting.
I20260621 07:59:24.582563 28025 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Bootstrap starting.
I20260621 07:59:24.583495 28023 tablet_bootstrap.cc:654] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:24.584424 28023 log.cc:824] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:24.584587 28025 tablet_bootstrap.cc:654] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:24.585063 28024 tablet_bootstrap.cc:654] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:24.585806 28025 log.cc:824] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:24.585817 28024 log.cc:824] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:24.586587 28023 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: No bootstrap required, opened a new log
I20260621 07:59:24.586802 28023 ts_tablet_manager.cc:1397] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 07:59:24.587553 28025 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: No bootstrap required, opened a new log
I20260621 07:59:24.587764 28025 ts_tablet_manager.cc:1397] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 07:59:24.589545 28024 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: No bootstrap required, opened a new log
I20260621 07:59:24.589722 28024 ts_tablet_manager.cc:1397] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260621 07:59:24.590372 28023 raft_consensus.cc:348] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.590607 28023 raft_consensus.cc:374] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:24.590693 28023 raft_consensus.cc:729] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8aa6a16e5b44069db84e1db26d4832, State: Initialized, Role: FOLLOWER
I20260621 07:59:24.591192 28023 consensus_queue.cc:260] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.591863 27738 heartbeater.cc:503] Master 127.11.223.190:32927 was elected leader, sending a full tablet report...
I20260621 07:59:24.592175 28023 ts_tablet_manager.cc:1428] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 07:59:24.591477 28025 raft_consensus.cc:348] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.592552 28025 raft_consensus.cc:374] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:24.592669 28025 raft_consensus.cc:729] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bac26543b0e4662a779fcd33e0c376b, State: Initialized, Role: FOLLOWER
I20260621 07:59:24.593199 28025 consensus_queue.cc:260] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.593405 28024 raft_consensus.cc:348] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.593820 28024 raft_consensus.cc:374] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:24.593919 28024 raft_consensus.cc:729] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4965d9216c0d426182adbd09fd3c8e68, State: Initialized, Role: FOLLOWER
I20260621 07:59:24.594173 28025 ts_tablet_manager.cc:1428] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260621 07:59:24.594354 28024 consensus_queue.cc:260] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.594935 27872 heartbeater.cc:503] Master 127.11.223.190:32927 was elected leader, sending a full tablet report...
I20260621 07:59:24.594970 28006 heartbeater.cc:503] Master 127.11.223.190:32927 was elected leader, sending a full tablet report...
I20260621 07:59:24.595197 28024 ts_tablet_manager.cc:1428] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260621 07:59:24.766705 28007 tablet.cc:2369] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:59:24.793457 27873 tablet.cc:2369] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 07:59:24.805570 27739 tablet.cc:2369] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:59:24.854044 28029 raft_consensus.cc:482] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:24.854249 28029 raft_consensus.cc:504] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.855355 28029 leader_election.cc:288] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4965d9216c0d426182adbd09fd3c8e68 (127.11.223.130:45779), 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971)
I20260621 07:59:24.859450 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.855602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:35971 (local address 127.11.223.129:41949)
0621 07:59:24.856030 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.856046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.856171 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.856495 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.856501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.856521 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:24.856665 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.856672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.857464 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.857467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.858308 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.858317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.858971 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.858976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.859083 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.859109 (+ 26us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.859205 (+ 96us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.859262 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":28,"thread_start_us":117,"threads_started":1}
I20260621 07:59:24.859457 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.855726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:41949 (local address 127.11.223.131:35971)
0621 07:59:24.855886 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.855890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.856156 (+ 266us) server_negotiation.cc:407] Connection header received
0621 07:59:24.856305 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.856335 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.856398 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.856456 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:24.856804 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.857327 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.858436 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.858850 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.858944 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.859007 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.859243 (+ 236us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.859333 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:59:24.859521 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.855916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:44803 (local address 127.11.223.130:45779)
0621 07:59:24.856062 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.856069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.856218 (+ 149us) server_negotiation.cc:407] Connection header received
0621 07:59:24.856399 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.856402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.856464 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.856541 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:24.856843 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.857323 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.858469 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.858923 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.859049 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.859097 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.859332 (+ 235us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.859401 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260621 07:59:24.859494 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.855602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:45779 (local address 127.11.223.129:44803)
0621 07:59:24.856092 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.856104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:24.856261 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:24.856570 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:24.856573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:24.856588 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:24.856699 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.856704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.857429 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.857433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.858323 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:24.858332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.859064 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.859071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:24.859170 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.859223 (+ 53us) client_negotiation.cc:769] Sending connection context
0621 07:59:24.859302 (+ 79us) client_negotiation.cc:240] Negotiation successful
0621 07:59:24.859362 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":61,"thread_start_us":87,"threads_started":1}
I20260621 07:59:24.860267 27960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff36a869943d40aa9a7733b7d6181746" candidate_uuid: "2a8aa6a16e5b44069db84e1db26d4832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bac26543b0e4662a779fcd33e0c376b" is_pre_election: true
I20260621 07:59:24.860364 27826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff36a869943d40aa9a7733b7d6181746" candidate_uuid: "2a8aa6a16e5b44069db84e1db26d4832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4965d9216c0d426182adbd09fd3c8e68" is_pre_election: true
I20260621 07:59:24.860630 27960 raft_consensus.cc:2454] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a8aa6a16e5b44069db84e1db26d4832 in term 0.
I20260621 07:59:24.860663 27826 raft_consensus.cc:2454] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a8aa6a16e5b44069db84e1db26d4832 in term 0.
I20260621 07:59:24.861171 27628 leader_election.cc:302] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2a8aa6a16e5b44069db84e1db26d4832, 4965d9216c0d426182adbd09fd3c8e68, 8bac26543b0e4662a779fcd33e0c376b; no voters:
I20260621 07:59:24.861456 28029 raft_consensus.cc:2781] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:59:24.861526 28029 raft_consensus.cc:482] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:59:24.861555 28029 raft_consensus.cc:3037] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:24.862629 28029 raft_consensus.cc:504] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.863003 28029 leader_election.cc:288] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [CANDIDATE]: Term 1 election: Requested vote from peers 4965d9216c0d426182adbd09fd3c8e68 (127.11.223.130:45779), 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971)
I20260621 07:59:24.863389 27960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff36a869943d40aa9a7733b7d6181746" candidate_uuid: "2a8aa6a16e5b44069db84e1db26d4832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bac26543b0e4662a779fcd33e0c376b"
I20260621 07:59:24.863389 27826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff36a869943d40aa9a7733b7d6181746" candidate_uuid: "2a8aa6a16e5b44069db84e1db26d4832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4965d9216c0d426182adbd09fd3c8e68"
I20260621 07:59:24.863515 27826 raft_consensus.cc:3037] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:24.863515 27960 raft_consensus.cc:3037] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:24.864796 27826 raft_consensus.cc:2454] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a8aa6a16e5b44069db84e1db26d4832 in term 1.
I20260621 07:59:24.864796 27960 raft_consensus.cc:2454] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a8aa6a16e5b44069db84e1db26d4832 in term 1.
I20260621 07:59:24.865118 27628 leader_election.cc:302] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2a8aa6a16e5b44069db84e1db26d4832, 4965d9216c0d426182adbd09fd3c8e68, 8bac26543b0e4662a779fcd33e0c376b; no voters:
I20260621 07:59:24.865257 28029 raft_consensus.cc:2781] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:24.865461 28029 raft_consensus.cc:686] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 LEADER]: Becoming Leader. State: Replica: 2a8aa6a16e5b44069db84e1db26d4832, State: Running, Role: LEADER
I20260621 07:59:24.865790 28029 consensus_queue.cc:237] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:24.868943 27544 catalog_manager.cc:5495] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129). New cstate: current_term: 1 leader_uuid: "2a8aa6a16e5b44069db84e1db26d4832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } health_report { overall_health: UNKNOWN } } }
I20260621 07:59:24.933806 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.930006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.11.223.129:36015)
0621 07:59:24.930158 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:24.930162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:24.930178 (+ 16us) server_negotiation.cc:407] Connection header received
0621 07:59:24.930292 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:24.930295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:24.930353 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:24.930473 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:59:24.930754 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.931259 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.932120 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:24.932359 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:24.932472 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:24.932896 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:59:24.933346 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:59:24.933491 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:24.933595 (+ 104us) server_negotiation.cc:299] Negotiation successful
0621 07:59:24.933658 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260621 07:59:24.940289 27960 raft_consensus.cc:1261] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Refusing update from remote peer 2a8aa6a16e5b44069db84e1db26d4832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:24.940289 27826 raft_consensus.cc:1261] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Refusing update from remote peer 2a8aa6a16e5b44069db84e1db26d4832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:24.941087 28034 consensus_queue.cc:1035] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:24.941323 28029 consensus_queue.cc:1035] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:24.951192 28039 mvcc.cc:205] Tried to move back new op lower bound from 7299189821186723840 to 7299189820892676096. Current Snapshot: MvccSnapshot[applied={T|T < 7299189821186723840}]
I20260621 07:59:24.957739 27546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:24.957896 27546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:24.960409 27546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Sending DeleteTablet for 3 replicas of tablet ff36a869943d40aa9a7733b7d6181746
I20260621 07:59:24.961299 27940 tablet_service.cc:1514] Processing DeleteTablet for tablet ff36a869943d40aa9a7733b7d6181746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:39404
I20260621 07:59:24.961341 27806 tablet_service.cc:1514] Processing DeleteTablet for tablet ff36a869943d40aa9a7733b7d6181746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50332
I20260621 07:59:24.961503 27672 tablet_service.cc:1514] Processing DeleteTablet for tablet ff36a869943d40aa9a7733b7d6181746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:46114
I20260621 07:59:24.961984 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27609
I20260621 07:59:24.964790 28047 tablet_replica.cc:330] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: stopping tablet replica
I20260621 07:59:24.965095 28048 tablet_replica.cc:330] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: stopping tablet replica
I20260621 07:59:24.965166 28047 raft_consensus.cc:2229] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:24.965415 28048 raft_consensus.cc:2229] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:24.965595 28047 raft_consensus.cc:2258] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:24.965760 28048 raft_consensus.cc:2258] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:24.967085 28048 ts_tablet_manager.cc:1905] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:24.967095 28047 ts_tablet_manager.cc:1905] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:24.969740 28048 ts_tablet_manager.cc:1918] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:24.969841 28048 log.cc:1196] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/ff36a869943d40aa9a7733b7d6181746
W20260621 07:59:24.969792 27533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:36015 (error 108)
I20260621 07:59:24.969868 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27743
I20260621 07:59:24.970209 28048 ts_tablet_manager.cc:1939] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Deleting consensus metadata
I20260621 07:59:24.973059 28047 ts_tablet_manager.cc:1918] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:24.973170 28047 log.cc:1196] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/ff36a869943d40aa9a7733b7d6181746
I20260621 07:59:24.973641 28047 ts_tablet_manager.cc:1939] T ff36a869943d40aa9a7733b7d6181746 P 8bac26543b0e4662a779fcd33e0c376b: Deleting consensus metadata
I20260621 07:59:24.973980 28019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.973646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36015 (local address 127.0.0.1:46130)
0621 07:59:24.973781 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.973861 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:36015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260621 07:59:24.974155 27533 catalog_manager.cc:4584] TS 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff36a869943d40aa9a7733b7d6181746: Network error: Client connection negotiation failed: client connection to 127.11.223.129:36015: connect: Connection refused (error 111)
I20260621 07:59:24.975044 27534 catalog_manager.cc:4841] TS 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971): tablet ff36a869943d40aa9a7733b7d6181746 (table test-table [id=a7385f15faf648c99e7cb805fe521a50]) successfully deleted
I20260621 07:59:24.975909 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27877
I20260621 07:59:24.977353 28019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:24.976035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:45779 (local address 127.0.0.1:50340)
0621 07:59:24.977195 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:24.977259 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.130:45779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1061}
I20260621 07:59:24.981998 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 27515
I20260621 07:59:44.988497 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32927
--webserver_interface=127.11.223.190
--webserver_port=41235
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 07:59:45.102222 28051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:45.102499 28051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:45.102566 28051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:45.107318 28051 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260621 07:59:45.107374 28051 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260621 07:59:45.107391 28051 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260621 07:59:45.107405 28051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 07:59:45.107450 28051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:45.107481 28051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:45.107513 28051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 07:59:45.107543 28051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 07:59:45.113381 28051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=41235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:45.114513 28051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:45.115695 28051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:45.129781 28058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:45.129779 28057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.129983 28051 server_base.cc:1034] running on GCE node
W20260621 07:59:45.129776 28060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.130710 28051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:45.131832 28051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:45.133023 28051 hybrid_clock.cc:648] HybridClock initialized: now 1782028785132988 us; error 49 us; skew 500 ppm
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:45.136091 28051 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 07:59:45.137122 28051 webserver.cc:460] Webserver started at http://127.11.223.190:41235/ using document root <none> and password file <none>
I20260621 07:59:45.137722 28051 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:45.137853 28051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:45.142372 28051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260621 07:59:45.144549 28067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.145885 28051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:45.146070 28051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "681ad4cedcfa42da9f31c0e950cc2369"
format_stamp: "Formatted at 2026-06-21 07:59:23 on dist-test-slave-0pn3"
server_key: "00844342186bd83f991da4c0c2198c35"
server_key_iv: "006f069e23b5f7451231bba04fb42c6d"
server_key_version: "encryptionkey@0"
I20260621 07:59:45.146690 28051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:45.170104 28051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:45.173189 28051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:45.173429 28051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:45.181020 28051 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:32927
I20260621 07:59:45.181317 28119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:32927 every 8 connection(s)
I20260621 07:59:45.182188 28051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 07:59:45.185230 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28051
I20260621 07:59:45.185724 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:36015
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:32927
--webserver_port=42083
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:45.187388 28120 sys_catalog.cc:263] Verifying existing consensus state
I20260621 07:59:45.189257 28120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap starting.
I20260621 07:59:45.206698 28120 log.cc:824] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:45.215428 28120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:59:45.215839 28120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap complete.
I20260621 07:59:45.220660 28120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:45.221063 28120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681ad4cedcfa42da9f31c0e950cc2369, State: Initialized, Role: FOLLOWER
I20260621 07:59:45.221619 28120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:45.221740 28120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 07:59:45.221835 28120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 07:59:45.221956 28120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 1 FOLLOWER]: Advancing to term 2
I20260621 07:59:45.223855 28120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:45.224221 28120 leader_election.cc:302] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 681ad4cedcfa42da9f31c0e950cc2369; no voters:
I20260621 07:59:45.224566 28120 leader_election.cc:288] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260621 07:59:45.224710 28124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Leader election won for term 2
I20260621 07:59:45.224962 28124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 LEADER]: Becoming Leader. State: Replica: 681ad4cedcfa42da9f31c0e950cc2369, State: Running, Role: LEADER
I20260621 07:59:45.225318 28124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 07:59:45.225759 28120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 07:59:45.226294 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 681ad4cedcfa42da9f31c0e950cc2369. Latest consensus state: current_term: 2 leader_uuid: "681ad4cedcfa42da9f31c0e950cc2369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } } }
I20260621 07:59:45.226401 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:45.226838 28125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "681ad4cedcfa42da9f31c0e950cc2369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } } }
I20260621 07:59:45.227016 28128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 07:59:45.227013 28125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: This master's current role is: LEADER
I20260621 07:59:45.231928 28128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 07:59:45.232666 28128 catalog_manager.cc:1259] Loaded cluster ID: 5357fa4dfebb4cc490ff29d74c0a6cbe
I20260621 07:59:45.232740 28128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 07:59:45.235212 28128 catalog_manager.cc:1504] Loading token signing keys...
I20260621 07:59:45.235970 28128 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Loaded TSK: 0
I20260621 07:59:45.261421 28128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Generated new TSK 1
I20260621 07:59:45.261722 28128 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 07:59:45.303272 28122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:45.303612 28122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:45.303699 28122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:45.307870 28122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:45.307930 28122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:45.308022 28122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 07:59:45.313405 28122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:36015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=42083
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:45.314525 28122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:45.315580 28122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:45.325181 28147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:45.325215 28148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.325503 28122 server_base.cc:1034] running on GCE node
W20260621 07:59:45.325646 28150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.326159 28122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:45.326956 28122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:45.328157 28122 hybrid_clock.cc:648] HybridClock initialized: now 1782028785328123 us; error 66 us; skew 500 ppm
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:45.331578 28122 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 07:59:45.332682 28122 webserver.cc:460] Webserver started at http://127.11.223.129:42083/ using document root <none> and password file <none>
I20260621 07:59:45.333288 28122 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:45.333407 28122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:45.337590 28122 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:59:45.339970 28157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.341630 28122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260621 07:59:45.341766 28122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "2a8aa6a16e5b44069db84e1db26d4832"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "a71a3f8063914081bee34ab0408a1638"
server_key_iv: "78996aef8a9629b6cea655e11d8ce1f9"
server_key_version: "encryptionkey@0"
I20260621 07:59:45.342492 28122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:45.368054 28122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:45.371870 28122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:45.372063 28122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:45.372813 28122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:45.374104 28164 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260621 07:59:45.378297 28122 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 07:59:45.378374 28122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260621 07:59:45.378436 28122 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 07:59:45.380815 28122 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 07:59:45.380873 28122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:45.381196 28164 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap starting.
I20260621 07:59:45.391438 28122 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:36015
I20260621 07:59:45.391798 28271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:36015 every 8 connection(s)
I20260621 07:59:45.392599 28122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 07:59:45.393813 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28122
I20260621 07:59:45.394325 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:45779
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:32927
--webserver_port=40585
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 07:59:45.402688 28164 log.cc:824] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Log is configured to *not* fsync() on all Append() calls
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:45.410501 28164 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 07:59:45.410995 28164 tablet_bootstrap.cc:492] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap complete.
I20260621 07:59:45.412170 28164 ts_tablet_manager.cc:1397] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent bootstrapping tablet: real 0.031s user 0.018s sys 0.009s
I20260621 07:59:45.414959 28164 raft_consensus.cc:348] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:45.415494 28164 raft_consensus.cc:729] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8aa6a16e5b44069db84e1db26d4832, State: Initialized, Role: FOLLOWER
I20260621 07:59:45.415616 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.394968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:56365)
0621 07:59:45.396392 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.396450 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.397670 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.399249 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.399256 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.399610 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:45.400082 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.400096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.401542 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.401546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.402611 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.402622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.403567 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.403573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.403814 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.407619 (+ 3805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:45.407648 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:45.409622 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:45.412155 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.412161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.412175 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.412554 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.412951 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.412954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.412956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.413122 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.413779 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:45.413785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:45.414061 (+ 276us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.414248 (+ 187us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.414571 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":146,"threads_started":1}
I20260621 07:59:45.415963 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.396196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:56365 (local address 127.11.223.190:32927)
0621 07:59:45.396661 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.396678 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.396739 (+ 61us) server_negotiation.cc:407] Connection header received
0621 07:59:45.398261 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.398279 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.398804 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.399192 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:45.400304 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.401402 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.402751 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.403400 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.403584 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.409854 (+ 6270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:45.409860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:45.409876 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:45.409910 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:45.412026 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.412685 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.412691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.412696 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.412846 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.413229 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.413232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.413234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.413659 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:45.413786 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.414407 (+ 621us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.414741 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":132,"threads_started":1}
I20260621 07:59:45.415993 28164 consensus_queue.cc:260] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } }
I20260621 07:59:45.417052 28164 ts_tablet_manager.cc:1428] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260621 07:59:45.418642 28272 heartbeater.cc:347] Connected to a master server at 127.11.223.190:32927
I20260621 07:59:45.418865 28272 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:45.419353 28272 heartbeater.cc:511] Master 127.11.223.190:32927 requested a full tablet report, sending...
I20260621 07:59:45.421868 28084 ts_manager.cc:194] Registered new tserver with Master: 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015)
I20260621 07:59:45.426252 28084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:56365
I20260621 07:59:45.429409 28272 heartbeater.cc:503] Master 127.11.223.190:32927 was elected leader, sending a full tablet report...
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 07:59:45.435935 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.425183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36015 (local address 127.0.0.1:38046)
0621 07:59:45.425806 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.425851 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.425972 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.426736 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.426742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.427183 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:45.427307 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.427315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.428475 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.428480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.429123 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.429133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.429807 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.429811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.429931 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.430572 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:45.430586 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:45.432230 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:45.434076 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.434086 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.434091 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.434384 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.434711 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.434714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.434716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.434806 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.435268 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:45.435274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:45.435404 (+ 130us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.435543 (+ 139us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.435750 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":116,"threads_started":1}
I20260621 07:59:45.435971 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.425609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.11.223.129:36015)
0621 07:59:45.426231 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.426240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.426258 (+ 18us) server_negotiation.cc:407] Connection header received
0621 07:59:45.426314 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.426318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.426580 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.426762 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:45.427431 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.428275 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.429258 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.429684 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.429803 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.432402 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:45.432411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:45.432423 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:45.432457 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:45.433960 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.434507 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.434513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.434515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.434603 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.434897 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.434901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.434903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.435149 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:45.435295 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.435625 (+ 330us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.435818 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":530,"thread_start_us":109,"threads_started":1}
I20260621 07:59:45.437158 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet ff36a869943d40aa9a7733b7d6181746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:38046
I20260621 07:59:45.438077 28281 tablet_replica.cc:330] stopping tablet replica
I20260621 07:59:45.438414 28281 raft_consensus.cc:2229] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:45.438691 28281 raft_consensus.cc:2258] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:45.439836 28281 ts_tablet_manager.cc:1905] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:45.444455 28281 ts_tablet_manager.cc:1918] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:45.444564 28281 log.cc:1196] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal/wals/ff36a869943d40aa9a7733b7d6181746
I20260621 07:59:45.444855 28281 ts_tablet_manager.cc:1939] T ff36a869943d40aa9a7733b7d6181746 P 2a8aa6a16e5b44069db84e1db26d4832: Deleting consensus metadata
I20260621 07:59:45.446925 28070 catalog_manager.cc:4841] TS 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015): tablet ff36a869943d40aa9a7733b7d6181746 (table test-table [id=a7385f15faf648c99e7cb805fe521a50]) successfully deleted
W20260621 07:59:45.518488 28275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:45.518765 28275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:45.518824 28275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:45.523135 28275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:45.523202 28275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:45.523262 28275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 07:59:45.528960 28275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:45779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=40585
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:45.530066 28275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:45.531108 28275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 07:59:45.541200 28275 server_base.cc:1034] running on GCE node
W20260621 07:59:45.541141 28289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:45.541145 28286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:45.541153 28287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.541724 28275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:45.542307 28275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:45.543463 28275 hybrid_clock.cc:648] HybridClock initialized: now 1782028785543432 us; error 46 us; skew 500 ppm
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:45.546272 28275 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 07:59:45.547109 28275 webserver.cc:460] Webserver started at http://127.11.223.130:40585/ using document root <none> and password file <none>
I20260621 07:59:45.547751 28275 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:45.547873 28275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:45.552105 28275 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 07:59:45.554296 28296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.555545 28275 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:45.555642 28275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "4965d9216c0d426182adbd09fd3c8e68"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "2a2a6a97e19a5bc6e345f112cca35703"
server_key_iv: "feacebc4dd952c72f052627e1c2a97a4"
server_key_version: "encryptionkey@0"
I20260621 07:59:45.556156 28275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:45.575788 28275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:45.578735 28275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:45.578892 28275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:45.579473 28275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:45.580559 28303 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260621 07:59:45.584432 28303 ts_tablet_manager.cc:1852] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260621 07:59:45.584815 28303 ts_tablet_manager.cc:1905] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:45.588778 28303 ts_tablet_manager.cc:1918] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:45.588969 28303 ts_tablet_manager.cc:1939] T ff36a869943d40aa9a7733b7d6181746 P 4965d9216c0d426182adbd09fd3c8e68: Deleting consensus metadata
I20260621 07:59:45.589452 28275 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260621 07:59:45.589509 28275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.001s
I20260621 07:59:45.589564 28275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:45.589581 28275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.598589 28275 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:45779
I20260621 07:59:45.598621 28410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:45779 every 8 connection(s)
I20260621 07:59:45.599480 28275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 07:59:45.604753 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28275
I20260621 07:59:45.605162 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:35971
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:32927
--webserver_port=44429
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:45.614449 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.601389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:37285 (local address 127.11.223.190:32927)
0621 07:59:45.601548 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.601552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.602388 (+ 836us) server_negotiation.cc:407] Connection header received
0621 07:59:45.603380 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.603383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.603428 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.603501 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:45.604788 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.605475 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.607074 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.607489 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.607612 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.610345 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:45.610351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:45.610356 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:45.610393 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:45.612180 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.612731 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.612737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.612740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.612838 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.613178 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.613184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.613187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.613455 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:45.613569 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.614059 (+ 490us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.614219 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 07:59:45.615101 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.601759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:37285)
0621 07:59:45.602216 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.602260 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.603173 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.603666 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.603673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.604009 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:45.604633 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.604657 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.605616 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.605622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.606938 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.606948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.607632 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.607634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.607772 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.608435 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:45.608451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:45.610155 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:45.612308 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.612313 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.612365 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.612628 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.612948 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.612950 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.612951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.613076 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.613553 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:45.613558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:45.613807 (+ 249us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.613958 (+ 151us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.614207 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260621 07:59:45.616492 28411 heartbeater.cc:347] Connected to a master server at 127.11.223.190:32927
I20260621 07:59:45.616737 28411 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:45.617224 28411 heartbeater.cc:511] Master 127.11.223.190:32927 requested a full tablet report, sending...
I20260621 07:59:45.618247 28084 ts_manager.cc:194] Registered new tserver with Master: 4965d9216c0d426182adbd09fd3c8e68 (127.11.223.130:45779)
I20260621 07:59:45.618815 28084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:37285
W20260621 07:59:45.718755 28414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 07:59:45.719084 28414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 07:59:45.719166 28414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 07:59:45.723475 28414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 07:59:45.723562 28414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 07:59:45.723649 28414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 07:59:45.729264 28414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:35971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=44429
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 07:59:45.730383 28414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 07:59:45.731446 28414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 07:59:45.741262 28420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 07:59:45.741285 28419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.741458 28414 server_base.cc:1034] running on GCE node
W20260621 07:59:45.741602 28422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 07:59:45.742022 28414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 07:59:45.742671 28414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 07:59:45.743866 28414 hybrid_clock.cc:648] HybridClock initialized: now 1782028785743830 us; error 50 us; skew 500 ppm
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 07:59:45.746706 28414 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 07:59:45.747711 28414 webserver.cc:460] Webserver started at http://127.11.223.131:44429/ using document root <none> and password file <none>
I20260621 07:59:45.748407 28414 fs_manager.cc:359] Metadata directory not provided
I20260621 07:59:45.748486 28414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 07:59:45.752445 28414 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 07:59:45.754377 28429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.755662 28414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 07:59:45.755754 28414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "8bac26543b0e4662a779fcd33e0c376b"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "dc8c439ad71520b870a87d88c2615451"
server_key_iv: "34e96fe13ba51d6ab2771347c19e0525"
server_key_version: "encryptionkey@0"
I20260621 07:59:45.756285 28414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 07:59:45.772202 28414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 07:59:45.775136 28414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 07:59:45.775309 28414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 07:59:45.775873 28414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 07:59:45.776755 28414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 07:59:45.776803 28414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.776845 28414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 07:59:45.776866 28414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 07:59:45.786216 28414 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:35971
I20260621 07:59:45.786233 28542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:35971 every 8 connection(s)
I20260621 07:59:45.787243 28414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 07:59:45.792369 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28414
Jun 21 07:59:45 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028785, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 07:59:45.801537 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.793095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.11.223.190:32927)
0621 07:59:45.793352 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.793356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.793514 (+ 158us) server_negotiation.cc:407] Connection header received
0621 07:59:45.793664 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.793667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.793702 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.793766 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:45.794723 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.795221 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.796064 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.796305 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.796454 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.798014 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:45.798017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:45.798020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:45.798044 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:45.799845 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.800271 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.800276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.800278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.800390 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.800681 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.800685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.800686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.800871 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:45.801107 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.801239 (+ 132us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.801357 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
I20260621 07:59:45.802258 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.789088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:54881 (local address 127.11.223.190:32927)
0621 07:59:45.789241 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.789246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.790314 (+ 1068us) server_negotiation.cc:407] Connection header received
0621 07:59:45.791172 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.791175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.791223 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.791310 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:45.792613 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.793261 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.795050 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.795297 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.795413 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.798311 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:45.798315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:45.798318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:45.798360 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:45.799909 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.800490 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.800493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.800495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.800574 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.800943 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.800947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.800948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.801126 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:45.801240 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.801995 (+ 755us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.802106 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 07:59:45.803390 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.789492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:54881)
0621 07:59:45.790145 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.790199 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.790978 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.791487 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.791494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.791857 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0621 07:59:45.792441 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.792463 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.793430 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.793434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.794923 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.794932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.795458 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.795462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.795645 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.796278 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 07:59:45.796295 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 07:59:45.798146 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 07:59:45.800033 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.800039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.800063 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.800389 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.800671 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 07:59:45.800674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 07:59:45.800676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 07:59:45.800830 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 07:59:45.801241 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 07:59:45.801248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 07:59:45.801563 (+ 315us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.801950 (+ 387us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.802444 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":119,"threads_started":1}
I20260621 07:59:45.804566 28543 heartbeater.cc:347] Connected to a master server at 127.11.223.190:32927
I20260621 07:59:45.804807 28543 heartbeater.cc:464] Registering TS with master...
I20260621 07:59:45.805308 28543 heartbeater.cc:511] Master 127.11.223.190:32927 requested a full tablet report, sending...
I20260621 07:59:45.806289 28084 ts_manager.cc:194] Registered new tserver with Master: 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971)
I20260621 07:59:45.806851 28084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:54881
I20260621 07:59:45.815316 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 07:59:45.819316 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.815986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.11.223.190:32927)
0621 07:59:45.816111 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.816121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.816367 (+ 246us) server_negotiation.cc:407] Connection header received
0621 07:59:45.816520 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.816522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.816626 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.816721 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:59:45.816958 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.817454 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.818265 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.818468 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.818540 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.818829 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:59:45.818991 (+ 162us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0621 07:59:45.819212 (+ 221us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53658: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260621 07:59:45.819375 28276 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53658: token expired
W20260621 07:59:45.819754 28012 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.223.190:32927)
I20260621 07:59:45.826766 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.820133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.11.223.190:32927)
0621 07:59:45.820260 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.820263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.820274 (+ 11us) server_negotiation.cc:407] Connection header received
0621 07:59:45.820404 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.820407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.820450 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.820544 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0621 07:59:45.821110 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.821562 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.822304 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.822490 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.822561 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.823789 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 07:59:45.823792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 07:59:45.823794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 07:59:45.823815 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 07:59:45.825132 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.825601 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.825604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.825605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.825687 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 07:59:45.825917 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 07:59:45.825919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 07:59:45.825920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 07:59:45.826084 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 07:59:45.826182 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.826428 (+ 246us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.826547 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:59:45.829316 28084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 07:59:45.832612 28084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 07:59:45.840868 28206 tablet_service.cc:1467] Processing CreateTablet for tablet 1d1f3f219ca241768458078cb6f5f484 (DEFAULT_TABLE table=test-table [id=19560bf539a74ed58b3763f87504789a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:45.841611 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1f3f219ca241768458078cb6f5f484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:45.844635 28164 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap starting.
I20260621 07:59:45.845582 28164 tablet_bootstrap.cc:654] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:45.845593 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.839301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:45779 (local address 127.0.0.1:49314)
0621 07:59:45.839429 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.839446 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.839547 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.840706 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.840709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.840727 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:45.840851 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.840858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.842693 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.842697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.843816 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.843827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.845109 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.845113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.845225 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.845251 (+ 26us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.845337 (+ 86us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.845415 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260621 07:59:45.845432 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.839726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:35971 (local address 127.0.0.1:58932)
0621 07:59:45.840054 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.840071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.840166 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.841024 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.841028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.841048 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:45.841136 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.841141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.842425 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.842429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.843380 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.843389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.844956 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.844960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.845064 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.845120 (+ 56us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.845183 (+ 63us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.845263 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":36,"thread_start_us":118,"threads_started":1}
I20260621 07:59:45.845626 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.839683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.11.223.130:45779)
0621 07:59:45.840039 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.840052 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.840076 (+ 24us) server_negotiation.cc:407] Connection header received
0621 07:59:45.840162 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.840170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.840471 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.840730 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:45.840987 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.842573 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.843952 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.844990 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.845110 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.845261 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.845368 (+ 107us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.845465 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":153,"threads_started":1}
I20260621 07:59:45.846295 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.839913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.11.223.131:35971)
0621 07:59:45.840459 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.840472 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.840496 (+ 24us) server_negotiation.cc:407] Connection header received
0621 07:59:45.840571 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.840576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.840838 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.840989 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:45.841248 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.842303 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.843520 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.844545 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.844661 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.845567 (+ 906us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.845616 (+ 49us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.846163 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":150,"threads_started":1}
I20260621 07:59:45.847157 28164 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: No bootstrap required, opened a new log
I20260621 07:59:45.847250 28164 ts_tablet_manager.cc:1397] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260621 07:59:45.847728 28164 raft_consensus.cc:348] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.847852 28164 raft_consensus.cc:374] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:45.847890 28164 raft_consensus.cc:729] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8aa6a16e5b44069db84e1db26d4832, State: Initialized, Role: FOLLOWER
I20260621 07:59:45.847513 28345 tablet_service.cc:1467] Processing CreateTablet for tablet 1d1f3f219ca241768458078cb6f5f484 (DEFAULT_TABLE table=test-table [id=19560bf539a74ed58b3763f87504789a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:45.848008 28164 consensus_queue.cc:260] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.848562 28164 ts_tablet_manager.cc:1428] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260621 07:59:45.848937 28345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1f3f219ca241768458078cb6f5f484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:45.849406 28477 tablet_service.cc:1467] Processing CreateTablet for tablet 1d1f3f219ca241768458078cb6f5f484 (DEFAULT_TABLE table=test-table [id=19560bf539a74ed58b3763f87504789a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 07:59:45.850399 28477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1f3f219ca241768458078cb6f5f484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 07:59:45.853165 28303 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Bootstrap starting.
I20260621 07:59:45.855435 28303 tablet_bootstrap.cc:654] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:45.856246 28303 log.cc:824] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:45.858346 28557 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Bootstrap starting.
I20260621 07:59:45.858440 28303 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: No bootstrap required, opened a new log
I20260621 07:59:45.858620 28303 ts_tablet_manager.cc:1397] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260621 07:59:45.860921 28557 tablet_bootstrap.cc:654] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Neither blocks nor log segments found. Creating new log.
I20260621 07:59:45.861641 28557 log.cc:824] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Log is configured to *not* fsync() on all Append() calls
I20260621 07:59:45.862025 28303 raft_consensus.cc:348] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.862267 28303 raft_consensus.cc:374] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:45.862357 28303 raft_consensus.cc:729] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4965d9216c0d426182adbd09fd3c8e68, State: Initialized, Role: FOLLOWER
I20260621 07:59:45.862782 28303 consensus_queue.cc:260] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.863513 28411 heartbeater.cc:503] Master 127.11.223.190:32927 was elected leader, sending a full tablet report...
I20260621 07:59:45.863785 28303 ts_tablet_manager.cc:1428] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 07:59:45.863848 28557 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: No bootstrap required, opened a new log
I20260621 07:59:45.864034 28557 ts_tablet_manager.cc:1397] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 07:59:45.867440 28557 raft_consensus.cc:348] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.867676 28557 raft_consensus.cc:374] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 07:59:45.867712 28557 raft_consensus.cc:729] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bac26543b0e4662a779fcd33e0c376b, State: Initialized, Role: FOLLOWER
I20260621 07:59:45.868098 28557 consensus_queue.cc:260] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.868768 28543 heartbeater.cc:503] Master 127.11.223.190:32927 was elected leader, sending a full tablet report...
I20260621 07:59:45.868979 28557 ts_tablet_manager.cc:1428] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
W20260621 07:59:45.894219 28273 tablet.cc:2369] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 07:59:45.931613 28559 raft_consensus.cc:482] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 07:59:45.931854 28559 raft_consensus.cc:504] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.933085 28559 leader_election.cc:288] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971), 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015)
I20260621 07:59:45.937091 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.933321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:35971 (local address 127.11.223.130:43039)
0621 07:59:45.933554 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.933569 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.933687 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.933980 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.933983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.934000 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:45.934128 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.934138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.935021 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.935024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.935921 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.935930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.936661 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.936664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.936775 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.936797 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.936868 (+ 71us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.936935 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":38}
I20260621 07:59:45.937216 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.933478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:53115 (local address 127.11.223.129:36015)
0621 07:59:45.933591 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.933597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.933867 (+ 270us) server_negotiation.cc:407] Connection header received
0621 07:59:45.933986 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.933989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.934053 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.934152 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:45.934550 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.935098 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.936113 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.936640 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.936763 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.936924 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.937023 (+ 99us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.937093 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 07:59:45.937265 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.933321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36015 (local address 127.11.223.130:53115)
0621 07:59:45.933756 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:45.933770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 07:59:45.933872 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 07:59:45.934301 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 07:59:45.934303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 07:59:45.934323 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 07:59:45.934431 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.934437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.935216 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.935218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.935964 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 07:59:45.935972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.936775 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.936778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 07:59:45.936885 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.936909 (+ 24us) client_negotiation.cc:769] Sending connection context
0621 07:59:45.936985 (+ 76us) client_negotiation.cc:240] Negotiation successful
0621 07:59:45.937046 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":72,"thread_start_us":115,"threads_started":1}
I20260621 07:59:45.937758 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.933394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:43039 (local address 127.11.223.131:35971)
0621 07:59:45.933574 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.933578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.933637 (+ 59us) server_negotiation.cc:407] Connection header received
0621 07:59:45.933806 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.933812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.933871 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.933960 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 07:59:45.934299 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.934916 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.936045 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.936552 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.936647 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.936719 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.936897 (+ 178us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.936984 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260621 07:59:45.937767 28226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1f3f219ca241768458078cb6f5f484" candidate_uuid: "4965d9216c0d426182adbd09fd3c8e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a8aa6a16e5b44069db84e1db26d4832" is_pre_election: true
I20260621 07:59:45.938102 28226 raft_consensus.cc:2454] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4965d9216c0d426182adbd09fd3c8e68 in term 0.
I20260621 07:59:45.938323 28497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1f3f219ca241768458078cb6f5f484" candidate_uuid: "4965d9216c0d426182adbd09fd3c8e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bac26543b0e4662a779fcd33e0c376b" is_pre_election: true
I20260621 07:59:45.938632 28497 raft_consensus.cc:2454] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4965d9216c0d426182adbd09fd3c8e68 in term 0.
I20260621 07:59:45.938627 28299 leader_election.cc:302] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a8aa6a16e5b44069db84e1db26d4832, 4965d9216c0d426182adbd09fd3c8e68; no voters:
I20260621 07:59:45.938933 28559 raft_consensus.cc:2781] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 07:59:45.939105 28559 raft_consensus.cc:482] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 07:59:45.939206 28559 raft_consensus.cc:3037] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:45.940644 28559 raft_consensus.cc:504] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.941071 28559 leader_election.cc:288] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 1 election: Requested vote from peers 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971), 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015)
I20260621 07:59:45.941462 28497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1f3f219ca241768458078cb6f5f484" candidate_uuid: "4965d9216c0d426182adbd09fd3c8e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bac26543b0e4662a779fcd33e0c376b"
I20260621 07:59:45.941457 28226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1f3f219ca241768458078cb6f5f484" candidate_uuid: "4965d9216c0d426182adbd09fd3c8e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a8aa6a16e5b44069db84e1db26d4832"
I20260621 07:59:45.941612 28226 raft_consensus.cc:3037] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:45.941568 28497 raft_consensus.cc:3037] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 0 FOLLOWER]: Advancing to term 1
I20260621 07:59:45.942631 28497 raft_consensus.cc:2454] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4965d9216c0d426182adbd09fd3c8e68 in term 1.
I20260621 07:59:45.942660 28226 raft_consensus.cc:2454] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4965d9216c0d426182adbd09fd3c8e68 in term 1.
I20260621 07:59:45.942947 28300 leader_election.cc:302] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4965d9216c0d426182adbd09fd3c8e68, 8bac26543b0e4662a779fcd33e0c376b; no voters:
I20260621 07:59:45.943145 28559 raft_consensus.cc:2781] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 07:59:45.943354 28559 raft_consensus.cc:686] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 LEADER]: Becoming Leader. State: Replica: 4965d9216c0d426182adbd09fd3c8e68, State: Running, Role: LEADER
I20260621 07:59:45.943681 28559 consensus_queue.cc:237] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 07:59:45.947018 28084 catalog_manager.cc:5495] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4965d9216c0d426182adbd09fd3c8e68 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "4965d9216c0d426182adbd09fd3c8e68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } health_report { overall_health: UNKNOWN } } }
I20260621 07:59:45.972205 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:45.968573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.11.223.130:45779)
0621 07:59:45.968724 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0621 07:59:45.968728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 07:59:45.968742 (+ 14us) server_negotiation.cc:407] Connection header received
0621 07:59:45.968917 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 07:59:45.968920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 07:59:45.968982 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 07:59:45.969100 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 07:59:45.969335 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.969867 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.970708 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 07:59:45.970934 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 07:59:45.971045 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 07:59:45.971361 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 07:59:45.971789 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 07:59:45.971926 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 07:59:45.971994 (+ 68us) server_negotiation.cc:299] Negotiation successful
0621 07:59:45.972052 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 07:59:45.978662 28497 raft_consensus.cc:1261] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Refusing update from remote peer 4965d9216c0d426182adbd09fd3c8e68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:45.978662 28226 raft_consensus.cc:1261] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Refusing update from remote peer 4965d9216c0d426182adbd09fd3c8e68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 07:59:45.979492 28559 consensus_queue.cc:1035] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:45.979714 28562 consensus_queue.cc:1035] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 07:59:45.987041 28566 mvcc.cc:205] Tried to move back new op lower bound from 7299189907359764480 to 7299189907228295168. Current Snapshot: MvccSnapshot[applied={T|T < 7299189907359764480}]
I20260621 07:59:45.987605 28567 mvcc.cc:205] Tried to move back new op lower bound from 7299189907359764480 to 7299189907228295168. Current Snapshot: MvccSnapshot[applied={T|T < 7299189907359764480}]
I20260621 07:59:45.996533 28084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:45.996734 28084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 07:59:45.999259 28084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Sending DeleteTablet for 3 replicas of tablet 1d1f3f219ca241768458078cb6f5f484
I20260621 07:59:46.000049 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d1f3f219ca241768458078cb6f5f484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:45 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:38046
I20260621 07:59:46.000065 28477 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d1f3f219ca241768458078cb6f5f484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:45 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:58932
I20260621 07:59:46.000150 28345 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d1f3f219ca241768458078cb6f5f484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 07:59:45 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:49314
I20260621 07:59:46.000723 28575 tablet_replica.cc:330] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: stopping tablet replica
I20260621 07:59:46.000849 28574 tablet_replica.cc:330] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: stopping tablet replica
I20260621 07:59:46.001005 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28122
I20260621 07:59:46.001044 28574 raft_consensus.cc:2229] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:46.001127 28575 raft_consensus.cc:2229] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 07:59:46.001636 28575 raft_consensus.cc:2258] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:46.004256 28575 ts_tablet_manager.cc:1905] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 07:59:46.007176 28575 ts_tablet_manager.cc:1918] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 07:59:46.007288 28575 log.cc:1196] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/1d1f3f219ca241768458078cb6f5f484
I20260621 07:59:46.007647 28575 ts_tablet_manager.cc:1939] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Deleting consensus metadata
W20260621 07:59:46.007637 28070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:36015 (error 108)
I20260621 07:59:46.007877 28576 tablet_replica.cc:330] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: stopping tablet replica
I20260621 07:59:46.008258 28576 raft_consensus.cc:2229] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 LEADER]: Raft consensus shutting down.
I20260621 07:59:46.008823 28576 raft_consensus.cc:2258] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 07:59:46.009094 28071 catalog_manager.cc:4841] TS 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971): tablet 1d1f3f219ca241768458078cb6f5f484 (table test-table [id=19560bf539a74ed58b3763f87504789a]) successfully deleted
I20260621 07:59:46.009611 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28275
I20260621 07:59:46.009637 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:46.009341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36015 (local address 127.0.0.1:38048)
0621 07:59:46.009468 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:46.009548 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:36015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260621 07:59:46.010246 28070 catalog_manager.cc:4584] TS 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d1f3f219ca241768458078cb6f5f484: Network error: Client connection negotiation failed: client connection to 127.11.223.129:36015: connect: Connection refused (error 111)
I20260621 07:59:46.015695 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28414
I20260621 07:59:46.017001 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 07:59:46.015907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:45779 (local address 127.0.0.1:49330)
0621 07:59:46.016816 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0621 07:59:46.016889 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.130:45779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":791}
I20260621 07:59:46.021103 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28051
I20260621 08:00:06.027606 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32927
--webserver_interface=127.11.223.190
--webserver_port=41235
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:06.140082 28578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:06.140427 28578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:06.140511 28578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:06.144601 28578 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260621 08:00:06.144688 28578 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260621 08:00:06.144734 28578 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260621 08:00:06.144769 28578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:06.144821 28578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:06.144896 28578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:06.144934 28578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:06.144968 28578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:06.150573 28578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=41235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:06.151748 28578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:06.152863 28578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:06.161736 28584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:06.161736 28583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:06.161736 28586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.161799 28578 server_base.cc:1034] running on GCE node
I20260621 08:00:06.162603 28578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:06.163740 28578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:06.164896 28578 hybrid_clock.cc:648] HybridClock initialized: now 1782028806164862 us; error 50 us; skew 500 ppm
Jun 21 08:00:06 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:06.171753 28578 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:06.172840 28578 webserver.cc:460] Webserver started at http://127.11.223.190:41235/ using document root <none> and password file <none>
I20260621 08:00:06.173454 28578 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:06.173517 28578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:06.177775 28578 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:06.180128 28593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:06.181476 28578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:06.181635 28578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "681ad4cedcfa42da9f31c0e950cc2369"
format_stamp: "Formatted at 2026-06-21 07:59:23 on dist-test-slave-0pn3"
server_key: "00844342186bd83f991da4c0c2198c35"
server_key_iv: "006f069e23b5f7451231bba04fb42c6d"
server_key_version: "encryptionkey@0"
I20260621 08:00:06.182228 28578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:06.198635 28578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:06.202095 28578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:06.202314 28578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:06.209774 28578 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:32927
I20260621 08:00:06.210057 28645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:32927 every 8 connection(s)
I20260621 08:00:06.210923 28578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:06.214473 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28578
I20260621 08:00:06.214911 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:36015
--local_ip_for_outbound_sockets=127.11.223.129
--tserver_master_addrs=127.11.223.190:32927
--webserver_port=42083
--webserver_interface=127.11.223.129
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:00:06.215384 28646 sys_catalog.cc:263] Verifying existing consensus state
I20260621 08:00:06.217128 28646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap starting.
I20260621 08:00:06.234436 28646 log.cc:824] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:06.245864 28646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 08:00:06.246276 28646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Bootstrap complete.
I20260621 08:00:06.250376 28646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 08:00:06.250794 28646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681ad4cedcfa42da9f31c0e950cc2369, State: Initialized, Role: FOLLOWER
I20260621 08:00:06.251338 28646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 08:00:06.251482 28646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:06.251590 28646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:06.251729 28646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 2 FOLLOWER]: Advancing to term 3
I20260621 08:00:06.253746 28646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 08:00:06.254101 28646 leader_election.cc:302] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 681ad4cedcfa42da9f31c0e950cc2369; no voters:
I20260621 08:00:06.254428 28646 leader_election.cc:288] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260621 08:00:06.254567 28650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 3 FOLLOWER]: Leader election won for term 3
I20260621 08:00:06.254838 28650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [term 3 LEADER]: Becoming Leader. State: Replica: 681ad4cedcfa42da9f31c0e950cc2369, State: Running, Role: LEADER
I20260621 08:00:06.255239 28650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } }
I20260621 08:00:06.255738 28646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:06.256758 28651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "681ad4cedcfa42da9f31c0e950cc2369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } } }
I20260621 08:00:06.256896 28651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:06.257269 28652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 681ad4cedcfa42da9f31c0e950cc2369. Latest consensus state: current_term: 3 leader_uuid: "681ad4cedcfa42da9f31c0e950cc2369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681ad4cedcfa42da9f31c0e950cc2369" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 32927 } } }
I20260621 08:00:06.257423 28652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:06.257768 28659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:06.262058 28659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:06.262630 28659 catalog_manager.cc:1259] Loaded cluster ID: 5357fa4dfebb4cc490ff29d74c0a6cbe
I20260621 08:00:06.262710 28659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:06.264585 28659 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:06.265249 28659 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Loaded TSK: 0
I20260621 08:00:06.265338 28659 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 681ad4cedcfa42da9f31c0e950cc2369: Loaded TSK: 1
F20260621 08:00:06.265965 28659 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f37a87809ad google::LogMessage::Fail() at ??:0
@ 0x7f37a878388a google::LogMessage::SendToLog() at ??:0
@ 0x7f37a87804de google::LogMessage::Flush() at ??:0
@ 0x7f37a8781a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f37ab44418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f37ab43cf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f37b30e97bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f37b30c399c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f37b30c4178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f37b3105df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f37b312482b std::function<>::operator()() at ??:0
@ 0x7f37b30c3b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f37b30c4f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f37b30c1579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f37b3105640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f37aa9d23d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f37aa9d2c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f37aa9d43f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f37aa9c3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f37ac0076db start_thread at ??:0
@ 0x7f37a727e61f clone at ??:0
W20260621 08:00:06.360917 28648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:06.361223 28648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:06.361307 28648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:06.365393 28648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:06.365483 28648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:06.365621 28648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:06.371151 28648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:36015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=42083
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:06.372305 28648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:06.373468 28648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:06.383801 28694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.383975 28648 server_base.cc:1034] running on GCE node
W20260621 08:00:06.383808 28696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:06.383801 28693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.384802 28648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:06.385661 28648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:06.386881 28648 hybrid_clock.cc:648] HybridClock initialized: now 1782028806386849 us; error 57 us; skew 500 ppm
Jun 21 08:00:06 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:06.390602 28648 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 08:00:06.391933 28648 webserver.cc:460] Webserver started at http://127.11.223.129:42083/ using document root <none> and password file <none>
I20260621 08:00:06.392823 28648 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:06.392935 28648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:06.398499 28648 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260621 08:00:06.401648 28703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:06.403448 28648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:06.403599 28648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "2a8aa6a16e5b44069db84e1db26d4832"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "a71a3f8063914081bee34ab0408a1638"
server_key_iv: "78996aef8a9629b6cea655e11d8ce1f9"
server_key_version: "encryptionkey@0"
I20260621 08:00:06.404433 28648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:06.426870 28648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:06.430960 28648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:06.431190 28648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:06.431957 28648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:06.433342 28710 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260621 08:00:06.437678 28648 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 08:00:06.437770 28648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260621 08:00:06.437844 28648 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 08:00:06.440250 28648 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 08:00:06.440339 28648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:06.440713 28710 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap starting.
I20260621 08:00:06.451097 28648 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:36015
I20260621 08:00:06.451315 28817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:36015 every 8 connection(s)
I20260621 08:00:06.452541 28648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 08:00:06.460827 28710 log.cc:824] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:06.462430 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28648
I20260621 08:00:06.462965 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:45779
--local_ip_for_outbound_sockets=127.11.223.130
--tserver_master_addrs=127.11.223.190:32927
--webserver_port=40585
--webserver_interface=127.11.223.130
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:00:06.468835 28710 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 08:00:06.469239 28710 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Bootstrap complete.
I20260621 08:00:06.470677 28710 ts_tablet_manager.cc:1397] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent bootstrapping tablet: real 0.030s user 0.017s sys 0.011s
I20260621 08:00:06.475090 28710 raft_consensus.cc:348] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:06.475811 28710 raft_consensus.cc:729] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8aa6a16e5b44069db84e1db26d4832, State: Initialized, Role: FOLLOWER
I20260621 08:00:06.476493 28710 consensus_queue.cc:260] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:06.477806 28710 ts_tablet_manager.cc:1428] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260621 08:00:06.514541 28820 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:06.455048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:56209)
0621 08:00:06.455666 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.455730 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:06.456999 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:06.513751 (+ 56752us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":160,"threads_started":1}
W20260621 08:00:06.515195 28818 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:32927 (0 consecutive failures): Network error: Failed to ping master at 127.11.223.190:32927: Client connection negotiation failed: client connection to 127.11.223.190:32927: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260621 08:00:06.515962 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.515693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:58651)
0621 08:00:06.515826 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.515904 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 08:00:06.516703 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.516485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:44069)
0621 08:00:06.516601 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.516646 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260621 08:00:06.516932 28818 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260621 08:00:06.585115 28822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:06.585451 28822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:06.585516 28822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:06.589658 28822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:06.589725 28822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:06.589785 28822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:06.595875 28822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:45779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=40585
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:06.597065 28822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:06.598093 28822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:06.608000 28829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:06.608000 28828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.608129 28822 server_base.cc:1034] running on GCE node
W20260621 08:00:06.608214 28831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.608731 28822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:06.609436 28822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:06.610613 28822 hybrid_clock.cc:648] HybridClock initialized: now 1782028806610581 us; error 46 us; skew 500 ppm
Jun 21 08:00:06 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:06.613404 28822 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 08:00:06.614287 28822 webserver.cc:460] Webserver started at http://127.11.223.130:40585/ using document root <none> and password file <none>
I20260621 08:00:06.614900 28822 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:06.614995 28822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:06.618917 28822 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:06.621179 28838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:06.622361 28822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:06.622500 28822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "4965d9216c0d426182adbd09fd3c8e68"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "2a2a6a97e19a5bc6e345f112cca35703"
server_key_iv: "feacebc4dd952c72f052627e1c2a97a4"
server_key_version: "encryptionkey@0"
I20260621 08:00:06.623101 28822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:06.632882 28822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:06.635716 28822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:06.635922 28822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:06.636574 28822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:06.637620 28845 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260621 08:00:06.641829 28822 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260621 08:00:06.641906 28822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260621 08:00:06.642002 28822 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260621 08:00:06.643833 28822 ts_tablet_manager.cc:610] Registered 1 tablets
I20260621 08:00:06.643901 28822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260621 08:00:06.644157 28845 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Bootstrap starting.
I20260621 08:00:06.654428 28822 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:45779
I20260621 08:00:06.654507 28952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:45779 every 8 connection(s)
I20260621 08:00:06.655427 28822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 08:00:06.659330 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.657607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:42013)
0621 08:00:06.658196 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.658334 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":139,"threads_started":1}
W20260621 08:00:06.659807 28953 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:32927 (0 consecutive failures): Network error: Failed to ping master at 127.11.223.190:32927: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
I20260621 08:00:06.660487 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.660214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:50275)
0621 08:00:06.660354 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.660410 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260621 08:00:06.661176 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.660950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:44523)
0621 08:00:06.661068 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.661118 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260621 08:00:06.661373 28953 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260621 08:00:06.662855 28845 log.cc:824] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:06.663787 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28822
I20260621 08:00:06.664276 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:35971
--local_ip_for_outbound_sockets=127.11.223.131
--tserver_master_addrs=127.11.223.190:32927
--webserver_port=44429
--webserver_interface=127.11.223.131
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:00:06.670343 28845 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 08:00:06.670707 28845 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Bootstrap complete.
I20260621 08:00:06.671852 28845 ts_tablet_manager.cc:1397] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Time spent bootstrapping tablet: real 0.028s user 0.016s sys 0.008s
I20260621 08:00:06.674595 28845 raft_consensus.cc:348] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:06.675064 28845 raft_consensus.cc:729] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4965d9216c0d426182adbd09fd3c8e68, State: Initialized, Role: FOLLOWER
I20260621 08:00:06.675518 28845 consensus_queue.cc:260] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:06.676627 28845 ts_tablet_manager.cc:1428] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260621 08:00:06.676846 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.676519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:49629)
0621 08:00:06.676694 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.676760 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260621 08:00:06.677531 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.677307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:60455)
0621 08:00:06.677427 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.677476 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260621 08:00:06.777217 28957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:06.777491 28957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:06.777534 28957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:06.781756 28957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:06.781818 28957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:06.781917 28957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:06.787281 28957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:35971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=44429
--tserver_master_addrs=127.11.223.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:06.788479 28957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:06.789554 28957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:06.799597 28964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.799724 28957 server_base.cc:1034] running on GCE node
W20260621 08:00:06.799602 28963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:06.799865 28966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:06.800305 28957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:06.800966 28957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:06.802136 28957 hybrid_clock.cc:648] HybridClock initialized: now 1782028806802104 us; error 46 us; skew 500 ppm
Jun 21 08:00:06 dist-test-slave-0pn3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:06.804874 28957 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 08:00:06.805826 28957 webserver.cc:460] Webserver started at http://127.11.223.131:44429/ using document root <none> and password file <none>
I20260621 08:00:06.806413 28957 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:06.806509 28957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:06.810706 28957 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:06.812841 28973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:06.814098 28957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:06.814239 28957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "8bac26543b0e4662a779fcd33e0c376b"
format_stamp: "Formatted at 2026-06-21 07:59:24 on dist-test-slave-0pn3"
server_key: "dc8c439ad71520b870a87d88c2615451"
server_key_iv: "34e96fe13ba51d6ab2771347c19e0525"
server_key_version: "encryptionkey@0"
I20260621 08:00:06.814822 28957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:06.843108 28957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:06.846326 28957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:06.846540 28957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:06.847174 28957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:06.847994 28957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:06.848073 28957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:06.848158 28957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:06.848206 28957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:06.857354 28957 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:35971
I20260621 08:00:06.857378 29086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:35971 every 8 connection(s)
I20260621 08:00:06.858403 28957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:06.861450 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 28957
I20260621 08:00:06.862715 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.860690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:46357)
0621 08:00:06.861205 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.861706 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":114,"threads_started":1}
W20260621 08:00:06.863178 29087 heartbeater.cc:644] Failed to heartbeat to 127.11.223.190:32927 (0 consecutive failures): Network error: Failed to ping master at 127.11.223.190:32927: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
I20260621 08:00:06.863763 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.863543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:51217)
0621 08:00:06.863656 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.863704 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260621 08:00:06.864359 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:06.864105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:55833)
0621 08:00:06.864218 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:06.864260 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260621 08:00:06.864530 29087 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260621 08:00:07.518071 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:07.517591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:59125)
0621 08:00:07.517847 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:07.517906 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
I20260621 08:00:07.678655 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:07.678249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:35177)
0621 08:00:07.678498 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:07.678562 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":68,"threads_started":1}
I20260621 08:00:07.865415 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:07.865051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:51499)
0621 08:00:07.865265 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:07.865314 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":65,"threads_started":1}
I20260621 08:00:08.117623 29094 raft_consensus.cc:482] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:08.117892 29094 raft_consensus.cc:504] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:08.119148 29094 leader_election.cc:288] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971), 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015)
Jun 21 08:00:08 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.131@KRBTEST.COM
Jun 21 08:00:08 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.129@KRBTEST.COM
I20260621 08:00:08.135139 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.119374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:35971 (local address 127.11.223.130:44235)
0621 08:00:08.120144 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.120158 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:08.120255 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:08.121833 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:08.121836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:08.122007 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:08.122098 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:08.122108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.123996 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.123999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:08.125700 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:08.125710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.126709 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.126713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:08.126992 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:08.128043 (+ 1051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:08.128054 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:08.129754 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:08.132100 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:08.132107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:08.132122 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:08.132487 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:08.132993 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:08.132995 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:08.132997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:08.133174 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:08.134112 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:08.134121 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:08.134409 (+ 288us) client_negotiation.cc:769] Sending connection context
0621 08:00:08.134626 (+ 217us) client_negotiation.cc:240] Negotiation successful
0621 08:00:08.134994 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":712,"spinlock_wait_cycles":10624,"thread_start_us":59,"threads_started":1}
I20260621 08:00:08.135262 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.119374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:36015 (local address 127.11.223.130:51833)
0621 08:00:08.119577 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.119633 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:08.120118 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:08.121084 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:08.121091 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:08.121473 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:08.122038 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:08.122057 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.123996 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.123999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:08.125700 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:08.125710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.126709 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.126713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:08.126994 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:08.127974 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:08.128018 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:08.130143 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:08.132510 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:08.132512 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:08.132514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:08.132885 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:08.133283 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:08.133286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:08.133287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:08.133403 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:08.134212 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:08.134215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:08.134406 (+ 191us) client_negotiation.cc:769] Sending connection context
0621 08:00:08.134625 (+ 219us) client_negotiation.cc:240] Negotiation successful
0621 08:00:08.135074 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":369,"spinlock_wait_cycles":768}
I20260621 08:00:08.135262 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.119537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:51833 (local address 127.11.223.129:36015)
0621 08:00:08.119941 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:08.119947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:08.119988 (+ 41us) server_negotiation.cc:407] Connection header received
0621 08:00:08.120508 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:08.120515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:08.120780 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:08.120999 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:08.122262 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.123836 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.125830 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.126567 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.126742 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:08.130308 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:08.130313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:08.130326 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:08.130354 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:08.132367 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:08.133006 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:08.133012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:08.133017 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:08.133180 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:08.133526 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:08.133528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:08.133529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:08.133996 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:08.134164 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:08.134808 (+ 644us) server_negotiation.cc:299] Negotiation successful
0621 08:00:08.135092 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":119,"threads_started":1}
I20260621 08:00:08.136065 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.119567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:44235 (local address 127.11.223.131:35971)
0621 08:00:08.120142 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:08.120162 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:08.120224 (+ 62us) server_negotiation.cc:407] Connection header received
0621 08:00:08.121216 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:08.121236 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:08.121608 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:08.121830 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:08.122264 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.123836 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.125830 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:08.126567 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:08.126742 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:08.130056 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:08.130065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:08.130083 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:08.130122 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:08.131957 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:08.132716 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:08.132724 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:08.132731 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:08.132884 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:08.133289 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:08.133293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:08.133294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:08.133894 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:08.134048 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:08.135611 (+ 1563us) server_negotiation.cc:299] Negotiation successful
0621 08:00:08.135895 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":102,"threads_started":1}
I20260621 08:00:08.136838 28772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1f3f219ca241768458078cb6f5f484" candidate_uuid: "4965d9216c0d426182adbd09fd3c8e68" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2a8aa6a16e5b44069db84e1db26d4832" is_pre_election: true
I20260621 08:00:08.137218 28772 raft_consensus.cc:2454] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4965d9216c0d426182adbd09fd3c8e68 in term 1.
W20260621 08:00:08.138414 28842 leader_election.cc:341] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971): Not found: Tablet not found: 1d1f3f219ca241768458078cb6f5f484
I20260621 08:00:08.138550 28842 leader_election.cc:302] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a8aa6a16e5b44069db84e1db26d4832, 4965d9216c0d426182adbd09fd3c8e68; no voters: 8bac26543b0e4662a779fcd33e0c376b
I20260621 08:00:08.138823 29094 raft_consensus.cc:2781] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260621 08:00:08.138900 29094 raft_consensus.cc:482] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:08.138940 29094 raft_consensus.cc:3037] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 1 FOLLOWER]: Advancing to term 2
I20260621 08:00:08.142750 29094 raft_consensus.cc:504] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:08.143203 29094 leader_election.cc:288] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 2 election: Requested vote from peers 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971), 2a8aa6a16e5b44069db84e1db26d4832 (127.11.223.129:36015)
I20260621 08:00:08.143778 28772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1f3f219ca241768458078cb6f5f484" candidate_uuid: "4965d9216c0d426182adbd09fd3c8e68" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2a8aa6a16e5b44069db84e1db26d4832"
I20260621 08:00:08.143939 28772 raft_consensus.cc:3037] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 1 FOLLOWER]: Advancing to term 2
W20260621 08:00:08.144069 28842 leader_election.cc:341] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971): Not found: Tablet not found: 1d1f3f219ca241768458078cb6f5f484
I20260621 08:00:08.146265 28772 raft_consensus.cc:2454] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4965d9216c0d426182adbd09fd3c8e68 in term 2.
I20260621 08:00:08.146636 28841 leader_election.cc:302] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a8aa6a16e5b44069db84e1db26d4832, 4965d9216c0d426182adbd09fd3c8e68; no voters: 8bac26543b0e4662a779fcd33e0c376b
I20260621 08:00:08.146863 29094 raft_consensus.cc:2781] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 2 FOLLOWER]: Leader election won for term 2
I20260621 08:00:08.147059 29094 raft_consensus.cc:686] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [term 2 LEADER]: Becoming Leader. State: Replica: 4965d9216c0d426182adbd09fd3c8e68, State: Running, Role: LEADER
I20260621 08:00:08.147416 29094 consensus_queue.cc:237] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:08.147768 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.147513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:53557)
0621 08:00:08.147645 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.147702 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260621 08:00:08.519357 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.518897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:57865)
0621 08:00:08.519154 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.519217 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
I20260621 08:00:08.583158 28772 raft_consensus.cc:1261] T 1d1f3f219ca241768458078cb6f5f484 P 2a8aa6a16e5b44069db84e1db26d4832 [term 2 FOLLOWER]: Refusing update from remote peer 4965d9216c0d426182adbd09fd3c8e68: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260621 08:00:08.583940 29094 consensus_queue.cc:1035] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260621 08:00:08.585983 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.585668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:34627)
0621 08:00:08.585816 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.585890 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260621 08:00:08.588434 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.588054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:47657)
0621 08:00:08.588274 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.588357 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260621 08:00:08.588748 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.588395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:47805)
0621 08:00:08.588623 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.588694 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260621 08:00:08.589068 28842 consensus_peers.cc:487] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68 -> Peer 8bac26543b0e4662a779fcd33e0c376b (127.11.223.131:35971): Couldn't send request to peer 8bac26543b0e4662a779fcd33e0c376b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d1f3f219ca241768458078cb6f5f484. This is attempt 1: this message will repeat every 5th retry.
I20260621 08:00:08.866506 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:08.866116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:49503)
0621 08:00:08.866362 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:08.866417 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260621 08:00:09.001183 29107 ts_tablet_manager.cc:927] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Initiating tablet copy from peer 4965d9216c0d426182adbd09fd3c8e68 (127.11.223.130:45779)
I20260621 08:00:09.001787 29107 tablet_copy_client.cc:316] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: tablet copy: Beginning tablet copy session from remote peer at address 127.11.223.130:45779
Jun 21 08:00:09 dist-test-slave-0pn3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028806, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.130@KRBTEST.COM
I20260621 08:00:09.011740 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.002285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:45779 (local address 127.11.223.131:51837)
0621 08:00:09.002440 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:09.002459 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:09.002597 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:09.003104 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:09.003109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:09.003502 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:09.003630 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:09.003637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:09.004876 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:09.004879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:09.005544 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:09.005552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:09.006089 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:09.006091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:09.006166 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:09.006697 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:09.006709 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:09.008191 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:09.010056 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:09.010098 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:09.010109 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:09.010366 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:09.010631 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:09.010633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:09.010634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:09.010713 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:09.011135 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:09.011140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:09.011288 (+ 148us) client_negotiation.cc:769] Sending connection context
0621 08:00:09.011424 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 08:00:09.011557 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260621 08:00:09.011801 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.002410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:51837 (local address 127.11.223.130:45779)
0621 08:00:09.002669 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:09.002701 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:09.002723 (+ 22us) server_negotiation.cc:407] Connection header received
0621 08:00:09.002769 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:09.002773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:09.002950 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:09.003075 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:09.003780 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:09.004773 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:09.005646 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:09.005991 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:09.006073 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:09.008375 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:09.008384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:09.008395 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:09.008420 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:09.009935 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:09.010464 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:09.010470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:09.010472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:09.010550 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:09.010787 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:09.010790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:09.010791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:09.011037 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:09.011139 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:09.011509 (+ 370us) server_negotiation.cc:299] Negotiation successful
0621 08:00:09.011633 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260621 08:00:09.012867 28927 tablet_copy_service.cc:140] P 4965d9216c0d426182adbd09fd3c8e68: Received BeginTabletCopySession request for tablet 1d1f3f219ca241768458078cb6f5f484 from peer 8bac26543b0e4662a779fcd33e0c376b ({username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:51837)
I20260621 08:00:09.013012 28927 tablet_copy_service.cc:161] P 4965d9216c0d426182adbd09fd3c8e68: Beginning new tablet copy session on tablet 1d1f3f219ca241768458078cb6f5f484 from peer 8bac26543b0e4662a779fcd33e0c376b at {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:51837: session id = 8bac26543b0e4662a779fcd33e0c376b-1d1f3f219ca241768458078cb6f5f484
I20260621 08:00:09.014168 28927 tablet_copy_source_session.cc:205] T 1d1f3f219ca241768458078cb6f5f484 P 4965d9216c0d426182adbd09fd3c8e68: Tablet Copy: opened 0 blocks and 1 log segments
I20260621 08:00:09.018287 29107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1f3f219ca241768458078cb6f5f484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:09.024536 29107 tablet_copy_client.cc:791] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: tablet copy: Starting download of 0 data blocks...
I20260621 08:00:09.024744 29107 tablet_copy_client.cc:655] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: tablet copy: Starting download of 1 WAL segments...
I20260621 08:00:09.026660 29107 tablet_copy_client.cc:531] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260621 08:00:09.028487 29107 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Bootstrap starting.
I20260621 08:00:09.033808 29107 log.cc:824] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:09.040176 29107 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260621 08:00:09.040514 29107 tablet_bootstrap.cc:492] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Bootstrap complete.
I20260621 08:00:09.040838 29107 ts_tablet_manager.cc:1397] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.003s
I20260621 08:00:09.043016 29107 raft_consensus.cc:348] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:09.043272 29107 raft_consensus.cc:729] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bac26543b0e4662a779fcd33e0c376b, State: Initialized, Role: FOLLOWER
I20260621 08:00:09.043705 29107 consensus_queue.cc:260] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4965d9216c0d426182adbd09fd3c8e68" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 45779 } } peers { permanent_uuid: "8bac26543b0e4662a779fcd33e0c376b" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35971 } } peers { permanent_uuid: "2a8aa6a16e5b44069db84e1db26d4832" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 36015 } }
I20260621 08:00:09.044693 29107 ts_tablet_manager.cc:1428] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260621 08:00:09.044950 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.044668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:34055)
0621 08:00:09.044829 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:09.044882 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260621 08:00:09.045449 28927 tablet_copy_service.cc:342] P 4965d9216c0d426182adbd09fd3c8e68: Request end of tablet copy session 8bac26543b0e4662a779fcd33e0c376b-1d1f3f219ca241768458078cb6f5f484 received from {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:51837
I20260621 08:00:09.045576 28927 tablet_copy_service.cc:433] P 4965d9216c0d426182adbd09fd3c8e68: ending tablet copy session 8bac26543b0e4662a779fcd33e0c376b-1d1f3f219ca241768458078cb6f5f484 on tablet 1d1f3f219ca241768458078cb6f5f484 with peer 8bac26543b0e4662a779fcd33e0c376b
I20260621 08:00:09.045722 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.045464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:34583)
0621 08:00:09.045580 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:09.045656 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 08:00:09.521893 29114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.521555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:53781)
0621 08:00:09.521754 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:09.521803 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":44,"threads_started":1}
I20260621 08:00:09.521890 29041 raft_consensus.cc:1203] T 1d1f3f219ca241768458078cb6f5f484 P 8bac26543b0e4662a779fcd33e0c376b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260621 08:00:09.522835 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.522601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:44481)
0621 08:00:09.522720 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:09.522777 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260621 08:00:09.589798 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:09.589447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:35129)
0621 08:00:09.589653 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:09.589707 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":63,"threads_started":1}
I20260621 08:00:10.523499 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:10.522910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:34689)
0621 08:00:10.523289 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:10.523371 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":66,"threads_started":1}
I20260621 08:00:10.524184 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:10.523473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:36951)
0621 08:00:10.524060 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:10.524110 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":66,"threads_started":1}
I20260621 08:00:10.591180 29118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:10.590722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:36577)
0621 08:00:10.591034 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:10.591088 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":83,"threads_started":1}
I20260621 08:00:11.524991 29119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:11.524526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:50659)
0621 08:00:11.524812 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:11.524877 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":107,"threads_started":1}
I20260621 08:00:11.525286 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:11.524867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:51899)
0621 08:00:11.525098 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:11.525182 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260621 08:00:11.592674 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:11.592150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:54301)
0621 08:00:11.592515 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:11.592578 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":72,"threads_started":1}
I20260621 08:00:12.526516 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:12.526013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:42391)
0621 08:00:12.526326 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:12.526392 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":116,"threads_started":1}
I20260621 08:00:12.526530 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:12.526117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:47589)
0621 08:00:12.526331 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:12.526392 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":58,"threads_started":1}
I20260621 08:00:12.527374 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:12.527142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:51001)
0621 08:00:12.527274 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:12.527327 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260621 08:00:12.593729 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:12.593366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:59759)
0621 08:00:12.593589 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:12.593641 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260621 08:00:13.527683 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:13.527303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:53349)
0621 08:00:13.527528 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:13.527579 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260621 08:00:13.528376 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:13.527989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:53033)
0621 08:00:13.528191 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:13.528248 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260621 08:00:13.594758 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:13.594400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:34509)
0621 08:00:13.594623 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:13.594670 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260621 08:00:14.528743 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:14.528274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:54407)
0621 08:00:14.528582 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:14.528653 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":152,"threads_started":1}
I20260621 08:00:14.529316 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:14.528985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:43501)
0621 08:00:14.529195 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:14.529241 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260621 08:00:14.595950 29130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:14.595539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:41047)
0621 08:00:14.595791 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:14.595840 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260621 08:00:15.529872 29131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:15.529428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:54741)
0621 08:00:15.529693 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:15.529765 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":68,"threads_started":1}
I20260621 08:00:15.530332 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:15.529927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:47775)
0621 08:00:15.530196 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:15.530256 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
I20260621 08:00:15.596997 29133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:15.596640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:51019)
0621 08:00:15.596852 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:15.596911 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":64,"threads_started":1}
I20260621 08:00:16.531163 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:16.530691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:53351)
0621 08:00:16.530971 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:16.531041 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260621 08:00:16.531409 29135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:16.530987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:53355)
0621 08:00:16.531228 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:16.531281 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":74,"threads_started":1}
I20260621 08:00:16.598218 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:16.597821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:43977)
0621 08:00:16.598072 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:16.598129 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":70,"threads_started":1}
I20260621 08:00:17.532596 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:17.532156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:35369)
0621 08:00:17.532431 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:17.532504 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":72,"threads_started":1}
I20260621 08:00:17.532728 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:17.532277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:47109)
0621 08:00:17.532529 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:17.532582 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":87,"threads_started":1}
I20260621 08:00:17.599321 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:17.598938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:60885)
0621 08:00:17.599165 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:17.599228 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
I20260621 08:00:18.533689 29140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:18.533215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:33677)
0621 08:00:18.533486 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:18.533572 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":107,"threads_started":1}
I20260621 08:00:18.533838 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:18.533377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:42485)
0621 08:00:18.533642 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:18.533708 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":76,"threads_started":1}
I20260621 08:00:18.600384 29142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:18.599952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:41135)
0621 08:00:18.600162 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:18.600209 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260621 08:00:19.534865 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:19.534467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:57015)
0621 08:00:19.534710 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:19.534762 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260621 08:00:19.534981 29144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:19.534586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:45223)
0621 08:00:19.534817 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:19.534865 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260621 08:00:19.601405 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:19.601020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:35215)
0621 08:00:19.601251 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:19.601313 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260621 08:00:20.536019 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:20.535539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:38857)
0621 08:00:20.535835 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:20.535909 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":103,"threads_started":1}
I20260621 08:00:20.536062 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:20.535659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:54033)
0621 08:00:20.535910 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:20.535962 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260621 08:00:20.602520 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:20.602062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:52625)
0621 08:00:20.602324 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:20.602377 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260621 08:00:21.537168 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:21.536781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.130:36701)
0621 08:00:21.537021 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:21.537075 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260621 08:00:21.537233 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:21.536781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.131:54667)
0621 08:00:21.537093 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:21.537143 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":87,"threads_started":1}
I20260621 08:00:21.603658 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:21.603279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:32927 (local address 127.11.223.129:39809)
0621 08:00:21.603504 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:21.603570 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":61,"threads_started":1}
I20260621 08:00:21.869031 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28648
I20260621 08:00:21.874392 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28822
I20260621 08:00:21.879632 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 28957
2026-06-21T08:00:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59417 ms)
[----------] 30 tests from SecurityITest (234466 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:21 dist-test-slave-0pn3 krb5kdc[29154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:21 dist-test-slave-0pn3 krb5kdc[29154](info): set up 2 sockets
Jun 21 08:00:21 dist-test-slave-0pn3 krb5kdc[29154](info): commencing operation
krb5kdc: starting...
W20260621 08:00:22.904306 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:22 dist-test-slave-0pn3 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:23.016630 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:34629
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:34629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:23.123893 29167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:23.124176 29167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:23.124259 29167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:23.128156 29167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:23.128237 29167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:23.128273 29167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:23.128320 29167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:23.128374 29167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:23.133323 29167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:34629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:34629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:23.134299 29167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:23.135380 29167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:23.143774 29175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:23.143769 29172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.143935 29167 server_base.cc:1034] running on GCE node
W20260621 08:00:23.144109 29173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.144704 29167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:23.145766 29167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:23.146943 29167 hybrid_clock.cc:648] HybridClock initialized: now 1782028823146918 us; error 50 us; skew 500 ppm
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:23.149703 29167 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:23.150496 29167 webserver.cc:460] Webserver started at http://127.11.223.190:40535/ using document root <none> and password file <none>
I20260621 08:00:23.150980 29167 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:23.151034 29167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:23.151185 29167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:23.153136 29167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "62ca340cad5bb2f2550ff1ebe6a9d37c"
server_key_iv: "d09d8bc5b51789e59a3274a4583ec2d7"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.153638 29167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "62ca340cad5bb2f2550ff1ebe6a9d37c"
server_key_iv: "d09d8bc5b51789e59a3274a4583ec2d7"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.157104 29167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 08:00:23.159293 29182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.160476 29167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:23.160569 29167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "62ca340cad5bb2f2550ff1ebe6a9d37c"
server_key_iv: "d09d8bc5b51789e59a3274a4583ec2d7"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.160651 29167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:23.169744 29167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:23.172773 29167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:23.172910 29167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:23.179626 29167 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:34629
I20260621 08:00:23.179651 29234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:34629 every 8 connection(s)
I20260621 08:00:23.180565 29167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:23.182085 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29167
I20260621 08:00:23.182206 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:23.182504 12158 external_mini_cluster.cc:1428] Setting key 48e01e26877198d87f25dbc1cc83f956
I20260621 08:00:23.183490 29235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:23.188871 29235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e: Bootstrap starting.
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:23.190974 29235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:23.191645 29235 log.cc:824] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:23.193693 29235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e: No bootstrap required, opened a new log
I20260621 08:00:23.196270 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.183878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.11.223.190:34629)
0621 08:00:23.184303 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.184363 (+ 60us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.184408 (+ 45us) server_negotiation.cc:407] Connection header received
0621 08:00:23.185034 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.185048 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.185383 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.185741 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:23.186528 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.187486 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.188344 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.188789 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.188955 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.191349 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:23.191355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:23.191366 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:23.191391 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:23.193274 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.193716 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.193722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.193729 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.193861 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.194159 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.194163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.194164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.194630 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:23.194819 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.195049 (+ 230us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.195315 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":146,"threads_started":1}
I20260621 08:00:23.197909 29235 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34629 } }
I20260621 08:00:23.198136 29235 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:23.198261 29235 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc14eda049a4dbabf5ce5ebbd1ac06e, State: Initialized, Role: FOLLOWER
I20260621 08:00:23.198697 29235 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34629 } }
I20260621 08:00:23.198815 29235 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:23.198894 29235 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:23.198988 29235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:23.199821 29235 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34629 } }
I20260621 08:00:23.200148 29235 leader_election.cc:302] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebc14eda049a4dbabf5ce5ebbd1ac06e; no voters:
I20260621 08:00:23.200439 29235 leader_election.cc:288] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:23.200554 29240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:23.200784 29240 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [term 1 LEADER]: Becoming Leader. State: Replica: ebc14eda049a4dbabf5ce5ebbd1ac06e, State: Running, Role: LEADER
I20260621 08:00:23.201134 29240 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34629 } }
I20260621 08:00:23.201635 29235 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:23.202685 29241 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34629 } } }
I20260621 08:00:23.202819 29241 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:23.203122 29242 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc14eda049a4dbabf5ce5ebbd1ac06e. Latest consensus state: current_term: 1 leader_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc14eda049a4dbabf5ce5ebbd1ac06e" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34629 } } }
I20260621 08:00:23.203202 29242 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:23.203210 29249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:23.206202 29249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:23.211334 29249 catalog_manager.cc:1347] Generated new cluster ID: 36b7348e6a92403c980380e6c142a266
I20260621 08:00:23.211395 29249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:23.241218 29249 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:23.242479 29249 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:23.256865 29249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e: Generated new TSK 0
I20260621 08:00:23.257570 29249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 08:00:23.293427 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:34629
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:23.400852 29261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:23.401070 29261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:23.401108 29261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:23.405391 29261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:23.405452 29261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:23.405562 29261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:23.410602 29261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:34629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:23.411885 29261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:23.413241 29261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:23.422482 29266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:23.422482 29269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:23.422592 29267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.422829 29261 server_base.cc:1034] running on GCE node
I20260621 08:00:23.423147 29261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:23.423744 29261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:23.424906 29261 hybrid_clock.cc:648] HybridClock initialized: now 1782028823424880 us; error 36 us; skew 500 ppm
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:23.427560 29261 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 08:00:23.428305 29261 webserver.cc:460] Webserver started at http://127.11.223.129:38801/ using document root <none> and password file <none>
I20260621 08:00:23.428912 29261 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:23.428973 29261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:23.429126 29261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:23.430955 29261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "7db80b4fe93546e68b7fa02efe9e52db"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "168e12959fade2a8247dc22ea3c5a455"
server_key_iv: "6e7dbb591345c76084b63430299feaf3"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.431432 29261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "7db80b4fe93546e68b7fa02efe9e52db"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "168e12959fade2a8247dc22ea3c5a455"
server_key_iv: "6e7dbb591345c76084b63430299feaf3"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.434825 29261 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260621 08:00:23.437036 29276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.438176 29261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 08:00:23.438313 29261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "7db80b4fe93546e68b7fa02efe9e52db"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "168e12959fade2a8247dc22ea3c5a455"
server_key_iv: "6e7dbb591345c76084b63430299feaf3"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.438421 29261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:23.452394 29261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:23.455583 29261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:23.455790 29261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:23.456403 29261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:23.457203 29261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:23.457273 29261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.457345 29261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:23.457394 29261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.466176 29261 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:39463
I20260621 08:00:23.466202 29389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:39463 every 8 connection(s)
I20260621 08:00:23.467059 29261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 08:00:23.469447 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29261
I20260621 08:00:23.469574 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 08:00:23.469915 12158 external_mini_cluster.cc:1428] Setting key 3ca438bfb587c8820e57e80489ef8e7f
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
I20260621 08:00:23.481036 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.468716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:52945 (local address 127.11.223.190:34629)
0621 08:00:23.468858 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.468862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.469641 (+ 779us) server_negotiation.cc:407] Connection header received
0621 08:00:23.471115 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.471118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.471170 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.471250 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:23.472344 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.473133 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.474280 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.474709 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.474834 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.477574 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:23.477577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:23.477580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:23.477600 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:23.478961 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.479520 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.479522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.479523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.479589 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.479933 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.479938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.479940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.480104 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:23.480205 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.480746 (+ 541us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.480875 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 08:00:23.481904 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.469014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:34629 (local address 127.11.223.129:52945)
0621 08:00:23.469483 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:23.469544 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:23.470915 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:23.471424 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:23.471431 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:23.471747 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:23.472177 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.472190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.473267 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.473270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.474156 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.474165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.474855 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.474858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.475002 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.475587 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:23.475602 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:23.477417 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:23.479081 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:23.479089 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:23.479105 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:23.479430 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:23.479697 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:23.479700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:23.479702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:23.479836 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:23.480228 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:23.480234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:23.480521 (+ 287us) client_negotiation.cc:769] Sending connection context
0621 08:00:23.480698 (+ 177us) client_negotiation.cc:240] Negotiation successful
0621 08:00:23.480984 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
I20260621 08:00:23.483152 29390 heartbeater.cc:347] Connected to a master server at 127.11.223.190:34629
I20260621 08:00:23.483410 29390 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:23.483995 29390 heartbeater.cc:511] Master 127.11.223.190:34629 requested a full tablet report, sending...
I20260621 08:00:23.485533 29199 ts_manager.cc:194] Registered new tserver with Master: 7db80b4fe93546e68b7fa02efe9e52db (127.11.223.129:39463)
I20260621 08:00:23.486651 29199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:52945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 08:00:23.498867 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:34629
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:23.603775 29395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:23.603983 29395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:23.604022 29395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:23.607903 29395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:23.607950 29395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:23.608036 29395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:23.613006 29395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:34629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:23.613933 29395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:23.614846 29395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:23.623802 29403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:23.623867 29400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.623929 29395 server_base.cc:1034] running on GCE node
W20260621 08:00:23.623811 29401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.624576 29395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:23.625092 29395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:23.626271 29395 hybrid_clock.cc:648] HybridClock initialized: now 1782028823626243 us; error 39 us; skew 500 ppm
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:23.629011 29395 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 08:00:23.629796 29395 webserver.cc:460] Webserver started at http://127.11.223.130:40989/ using document root <none> and password file <none>
I20260621 08:00:23.630304 29395 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:23.630362 29395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:23.630574 29395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:23.632490 29395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "421fbbeafd234188975210af11eee247"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "2f1084c2ad70efc4334f947d3b25a2a9"
server_key_iv: "8da355f536fa04b2f2c958b0856bbb42"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.633016 29395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "421fbbeafd234188975210af11eee247"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "2f1084c2ad70efc4334f947d3b25a2a9"
server_key_iv: "8da355f536fa04b2f2c958b0856bbb42"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.636303 29395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:23.638514 29410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.639665 29395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 08:00:23.639796 29395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "421fbbeafd234188975210af11eee247"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "2f1084c2ad70efc4334f947d3b25a2a9"
server_key_iv: "8da355f536fa04b2f2c958b0856bbb42"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.639899 29395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:23.665110 29395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:23.668413 29395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:23.668601 29395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:23.669160 29395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:23.669907 29395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:23.669976 29395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.670039 29395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:23.670090 29395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.679008 29395 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:39857
I20260621 08:00:23.679036 29523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:39857 every 8 connection(s)
I20260621 08:00:23.679818 29395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 08:00:23.684842 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29395
I20260621 08:00:23.684988 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 08:00:23.685381 12158 external_mini_cluster.cc:1428] Setting key 053aaee8875ac5ee1965be57110f8883
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:23.693707 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.681666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:57889 (local address 127.11.223.190:34629)
0621 08:00:23.681803 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.681806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.682537 (+ 731us) server_negotiation.cc:407] Connection header received
0621 08:00:23.683792 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.683795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.683839 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.683914 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:23.685354 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.686026 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.687269 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.687487 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.687597 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.689979 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:23.689985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:23.689992 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:23.690023 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:23.691530 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.692158 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.692160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.692162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.692233 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.692632 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.692635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.692637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.692806 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:23.692882 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.693498 (+ 616us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.693585 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 08:00:23.694790 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.681990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:34629 (local address 127.11.223.130:57889)
0621 08:00:23.682392 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:23.682431 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:23.683566 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:23.684060 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:23.684066 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:23.684478 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:23.685175 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.685198 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.686149 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.686153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.687134 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.687143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.687629 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.687633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.687802 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.688410 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:23.688425 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:23.689824 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:23.691649 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:23.691658 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:23.691676 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:23.692056 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:23.692349 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:23.692354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:23.692359 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:23.692532 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:23.692909 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:23.692917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:23.693221 (+ 304us) client_negotiation.cc:769] Sending connection context
0621 08:00:23.693455 (+ 234us) client_negotiation.cc:240] Negotiation successful
0621 08:00:23.693769 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 08:00:23.695911 29524 heartbeater.cc:347] Connected to a master server at 127.11.223.190:34629
I20260621 08:00:23.696130 29524 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:23.696647 29524 heartbeater.cc:511] Master 127.11.223.190:34629 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 08:00:23.697697 29199 ts_manager.cc:194] Registered new tserver with Master: 421fbbeafd234188975210af11eee247 (127.11.223.130:39857)
I20260621 08:00:23.698251 29199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:57889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 08:00:23.711455 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:34629
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:23.817601 29529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:23.817845 29529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:23.817919 29529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:23.822070 29529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:23.822119 29529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:23.822234 29529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:23.827886 29529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:34629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:23.828871 29529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:23.829804 29529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:23.838667 29534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:23.838682 29537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.838904 29529 server_base.cc:1034] running on GCE node
W20260621 08:00:23.838676 29535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:23.839327 29529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:23.839901 29529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:23.841045 29529 hybrid_clock.cc:648] HybridClock initialized: now 1782028823841012 us; error 45 us; skew 500 ppm
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:23.843803 29529 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 08:00:23.844830 29529 webserver.cc:460] Webserver started at http://127.11.223.131:42185/ using document root <none> and password file <none>
I20260621 08:00:23.845387 29529 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:23.845479 29529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:23.845674 29529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:23.847615 29529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "66e5293bc1e3451ab7fdde33556a595c"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "1e666e211292e4d776a9af2ffe749801"
server_key_iv: "b4630c8b9ed73fd00252817d01adb4c7"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.848143 29529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "66e5293bc1e3451ab7fdde33556a595c"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "1e666e211292e4d776a9af2ffe749801"
server_key_iv: "b4630c8b9ed73fd00252817d01adb4c7"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.851608 29529 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 08:00:23.853839 29544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.854957 29529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:23.855085 29529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "66e5293bc1e3451ab7fdde33556a595c"
format_stamp: "Formatted at 2026-06-21 08:00:23 on dist-test-slave-0pn3"
server_key: "1e666e211292e4d776a9af2ffe749801"
server_key_iv: "b4630c8b9ed73fd00252817d01adb4c7"
server_key_version: "encryptionkey@0"
I20260621 08:00:23.855188 29529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:23.881822 29529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:23.884970 29529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:23.885174 29529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:23.885713 29529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:23.886523 29529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:23.886592 29529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.886658 29529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:23.886711 29529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:23.895165 29529 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:33065
I20260621 08:00:23.895193 29657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:33065 every 8 connection(s)
I20260621 08:00:23.895960 29529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:23.896727 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29529
I20260621 08:00:23.896893 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 08:00:23.897225 12158 external_mini_cluster.cc:1428] Setting key 344c440b38b8cefd5c838505d45eb22b
Jun 21 08:00:23 dist-test-slave-0pn3 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:23.908857 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.897980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:57037 (local address 127.11.223.190:34629)
0621 08:00:23.898126 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.898131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.898840 (+ 709us) server_negotiation.cc:407] Connection header received
0621 08:00:23.900111 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.900113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.900157 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.900224 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:23.901405 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.901918 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.903146 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.903364 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.903440 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.905662 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:23.905665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:23.905667 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:23.905688 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:23.907119 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.907541 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.907543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.907545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.907614 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.907888 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.907890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.907891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.908040 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:23.908111 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.908647 (+ 536us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.908744 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260621 08:00:23.909587 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.898258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:34629 (local address 127.11.223.131:57037)
0621 08:00:23.898686 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:23.898735 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:23.899937 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:23.900436 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:23.900442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:23.900766 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:23.901268 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.901283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.902034 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.902037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.902984 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.902994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.903481 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.903484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.903615 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.904101 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:23.904116 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:23.905524 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:23.907231 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:23.907235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:23.907244 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:23.907460 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:23.907688 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:23.907690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:23.907691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:23.907815 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:23.908119 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:23.908123 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:23.908440 (+ 317us) client_negotiation.cc:769] Sending connection context
0621 08:00:23.908602 (+ 162us) client_negotiation.cc:240] Negotiation successful
0621 08:00:23.908815 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":118,"threads_started":1}
I20260621 08:00:23.910657 29658 heartbeater.cc:347] Connected to a master server at 127.11.223.190:34629
I20260621 08:00:23.910882 29658 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:23.911355 29658 heartbeater.cc:511] Master 127.11.223.190:34629 requested a full tablet report, sending...
I20260621 08:00:23.912256 29199 ts_manager.cc:194] Registered new tserver with Master: 66e5293bc1e3451ab7fdde33556a595c (127.11.223.131:33065)
I20260621 08:00:23.912811 29199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:57037
I20260621 08:00:23.922345 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 08:00:23.930817 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.924217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.11.223.190:34629)
0621 08:00:23.924372 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.924378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.924588 (+ 210us) server_negotiation.cc:407] Connection header received
0621 08:00:23.924671 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.924673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.924713 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.924778 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:23.925563 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.926045 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.926803 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.926993 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.927063 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.928372 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:23.928376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:23.928384 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:23.928402 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:23.929537 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.929892 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.929894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.929895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.929959 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:23.930168 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:23.930170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:23.930171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:23.930341 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:23.930432 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.930601 (+ 169us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.930677 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 08:00:23.933362 29199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 08:00:23.936190 29199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 08:00:23.949378 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.943887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:39857 (local address 127.0.0.1:39724)
0621 08:00:23.944450 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:23.944500 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:23.944628 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:23.945446 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:23.945452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:23.945474 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:23.945599 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.945604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.946906 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.946912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.947818 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.947828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.948920 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.948924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.949030 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.949067 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 08:00:23.949165 (+ 98us) client_negotiation.cc:240] Negotiation successful
0621 08:00:23.949276 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":66,"threads_started":1}
I20260621 08:00:23.949812 29670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.944035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39724 (local address 127.11.223.130:39857)
0621 08:00:23.944432 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.944444 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.944799 (+ 355us) server_negotiation.cc:407] Connection header received
0621 08:00:23.944889 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.944895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.945096 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.945263 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:23.945757 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.946794 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.947947 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.948801 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.948922 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.949060 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.949570 (+ 510us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.949679 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":134,"threads_started":1}
I20260621 08:00:23.950466 29673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.944786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39463 (local address 127.0.0.1:50088)
0621 08:00:23.945334 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:23.945348 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:23.945445 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:23.946106 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:23.946109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:23.946126 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:23.946232 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.946239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.948174 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.948179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.949062 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.949069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.950052 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.950055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.950158 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.950178 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 08:00:23.950238 (+ 60us) client_negotiation.cc:240] Negotiation successful
0621 08:00:23.950298 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":91,"threads_started":1}
I20260621 08:00:23.950650 29671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.945082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:33065 (local address 127.0.0.1:35956)
0621 08:00:23.945550 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:23.945567 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:23.945659 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:23.946378 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:23.946381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:23.946397 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:23.946494 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.946499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.947777 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.947780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.948928 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:23.948938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.950177 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.950180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:23.950272 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.950291 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 08:00:23.950350 (+ 59us) client_negotiation.cc:240] Negotiation successful
0621 08:00:23.950418 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":129,"threads_started":1}
I20260621 08:00:23.950789 29674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.945483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.11.223.131:33065)
0621 08:00:23.945806 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.945817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.945838 (+ 21us) server_negotiation.cc:407] Connection header received
0621 08:00:23.945904 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.945909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.946096 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.946244 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:23.946612 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.947630 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.949054 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.949865 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.950000 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.950124 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.950534 (+ 410us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.950636 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":111,"threads_started":1}
I20260621 08:00:23.951138 29672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:23.944948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.11.223.129:39463)
0621 08:00:23.945355 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:23.945372 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:23.945769 (+ 397us) server_negotiation.cc:407] Connection header received
0621 08:00:23.945836 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:23.945840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:23.945968 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:23.946118 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:23.946359 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.948052 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.949167 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:23.949928 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:23.950792 (+ 864us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:23.950929 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:23.950971 (+ 42us) server_negotiation.cc:299] Negotiation successful
0621 08:00:23.951038 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":79,"threads_started":1}
I20260621 08:00:23.952379 29458 tablet_service.cc:1467] Processing CreateTablet for tablet 71ece9ef732b41fcbbb35093fed34625 (DEFAULT_TABLE table=test-table [id=961d4a66571f4afdb13292c67ceb7b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:23.953418 29458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ece9ef732b41fcbbb35093fed34625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:23.954737 29592 tablet_service.cc:1467] Processing CreateTablet for tablet 71ece9ef732b41fcbbb35093fed34625 (DEFAULT_TABLE table=test-table [id=961d4a66571f4afdb13292c67ceb7b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:23.955433 29592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ece9ef732b41fcbbb35093fed34625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:23.955578 29324 tablet_service.cc:1467] Processing CreateTablet for tablet 71ece9ef732b41fcbbb35093fed34625 (DEFAULT_TABLE table=test-table [id=961d4a66571f4afdb13292c67ceb7b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:23.956549 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ece9ef732b41fcbbb35093fed34625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:23.960539 29675 tablet_bootstrap.cc:492] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Bootstrap starting.
I20260621 08:00:23.962301 29676 tablet_bootstrap.cc:492] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Bootstrap starting.
I20260621 08:00:23.962916 29677 tablet_bootstrap.cc:492] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: Bootstrap starting.
I20260621 08:00:23.963606 29675 tablet_bootstrap.cc:654] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:23.964356 29676 tablet_bootstrap.cc:654] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:23.964943 29677 tablet_bootstrap.cc:654] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:23.965013 29675 log.cc:824] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:23.965559 29676 log.cc:824] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:23.966140 29677 log.cc:824] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:23.967120 29675 tablet_bootstrap.cc:492] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: No bootstrap required, opened a new log
I20260621 08:00:23.967362 29675 ts_tablet_manager.cc:1397] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260621 08:00:23.967635 29676 tablet_bootstrap.cc:492] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: No bootstrap required, opened a new log
I20260621 08:00:23.967803 29676 ts_tablet_manager.cc:1397] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260621 08:00:23.968124 29677 tablet_bootstrap.cc:492] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: No bootstrap required, opened a new log
I20260621 08:00:23.968302 29677 ts_tablet_manager.cc:1397] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 08:00:23.970955 29677 raft_consensus.cc:348] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:23.971179 29677 raft_consensus.cc:374] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:23.971258 29677 raft_consensus.cc:729] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db80b4fe93546e68b7fa02efe9e52db, State: Initialized, Role: FOLLOWER
I20260621 08:00:23.971210 29675 raft_consensus.cc:348] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:23.971459 29675 raft_consensus.cc:374] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:23.971344 29676 raft_consensus.cc:348] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:23.971549 29676 raft_consensus.cc:374] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:23.971578 29675 raft_consensus.cc:729] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421fbbeafd234188975210af11eee247, State: Initialized, Role: FOLLOWER
I20260621 08:00:23.971611 29676 raft_consensus.cc:729] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e5293bc1e3451ab7fdde33556a595c, State: Initialized, Role: FOLLOWER
I20260621 08:00:23.971769 29677 consensus_queue.cc:260] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:23.972020 29675 consensus_queue.cc:260] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:23.972020 29676 consensus_queue.cc:260] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:23.972410 29390 heartbeater.cc:503] Master 127.11.223.190:34629 was elected leader, sending a full tablet report...
I20260621 08:00:23.972712 29677 ts_tablet_manager.cc:1428] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260621 08:00:23.972956 29676 ts_tablet_manager.cc:1428] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 08:00:23.972956 29675 ts_tablet_manager.cc:1428] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260621 08:00:23.973278 29658 heartbeater.cc:503] Master 127.11.223.190:34629 was elected leader, sending a full tablet report...
I20260621 08:00:23.974160 29524 heartbeater.cc:503] Master 127.11.223.190:34629 was elected leader, sending a full tablet report...
W20260621 08:00:24.146883 29659 tablet.cc:2369] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 08:00:24.180994 29525 tablet.cc:2369] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 08:00:24.218722 29391 tablet.cc:2369] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:24.233958 29683 raft_consensus.cc:482] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:24.234187 29683 raft_consensus.cc:504] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:24.235273 29683 leader_election.cc:288] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7db80b4fe93546e68b7fa02efe9e52db (127.11.223.129:39463), 66e5293bc1e3451ab7fdde33556a595c (127.11.223.131:33065)
I20260621 08:00:24.239042 29672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.235601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:53497 (local address 127.11.223.129:39463)
0621 08:00:24.235730 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:24.235734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:24.235851 (+ 117us) server_negotiation.cc:407] Connection header received
0621 08:00:24.236017 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:24.236023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:24.236085 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:24.236136 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:24.236546 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.237062 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.238104 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.238495 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.238620 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.238673 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:24.238846 (+ 173us) server_negotiation.cc:299] Negotiation successful
0621 08:00:24.238921 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 08:00:24.239059 29684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.235504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39463 (local address 127.11.223.130:53497)
0621 08:00:24.235761 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:24.235778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:24.235875 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:24.236258 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:24.236260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:24.236275 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:24.236434 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.236439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.237184 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.237188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.237981 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.237989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.238609 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.238613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.238724 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.238743 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 08:00:24.238831 (+ 88us) client_negotiation.cc:240] Negotiation successful
0621 08:00:24.238925 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":24,"thread_start_us":105,"threads_started":1}
I20260621 08:00:24.239197 29674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.235981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:37731 (local address 127.11.223.131:33065)
0621 08:00:24.236097 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:24.236104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:24.236169 (+ 65us) server_negotiation.cc:407] Connection header received
0621 08:00:24.236360 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:24.236366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:24.236420 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:24.236505 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:24.236799 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.237289 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.238299 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.238688 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.238814 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.238860 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:24.239031 (+ 171us) server_negotiation.cc:299] Negotiation successful
0621 08:00:24.239088 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 08:00:24.239210 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.235869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:33065 (local address 127.11.223.130:37731)
0621 08:00:24.236062 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:24.236075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:24.236217 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:24.236515 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:24.236518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:24.236534 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:24.236692 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.236698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.237403 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.237406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.238180 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.238188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.238807 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.238811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.238914 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.238932 (+ 18us) client_negotiation.cc:769] Sending connection context
0621 08:00:24.239019 (+ 87us) client_negotiation.cc:240] Negotiation successful
0621 08:00:24.239081 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":58,"threads_started":1}
I20260621 08:00:24.239818 29344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ece9ef732b41fcbbb35093fed34625" candidate_uuid: "421fbbeafd234188975210af11eee247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db80b4fe93546e68b7fa02efe9e52db" is_pre_election: true
I20260621 08:00:24.239825 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ece9ef732b41fcbbb35093fed34625" candidate_uuid: "421fbbeafd234188975210af11eee247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e5293bc1e3451ab7fdde33556a595c" is_pre_election: true
I20260621 08:00:24.240095 29612 raft_consensus.cc:2454] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 421fbbeafd234188975210af11eee247 in term 0.
I20260621 08:00:24.240095 29344 raft_consensus.cc:2454] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 421fbbeafd234188975210af11eee247 in term 0.
I20260621 08:00:24.240545 29411 leader_election.cc:302] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 421fbbeafd234188975210af11eee247, 66e5293bc1e3451ab7fdde33556a595c; no voters:
I20260621 08:00:24.240764 29683 raft_consensus.cc:2781] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 08:00:24.240844 29683 raft_consensus.cc:482] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:24.240885 29683 raft_consensus.cc:3037] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:24.241849 29683 raft_consensus.cc:504] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:24.242183 29683 leader_election.cc:288] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [CANDIDATE]: Term 1 election: Requested vote from peers 7db80b4fe93546e68b7fa02efe9e52db (127.11.223.129:39463), 66e5293bc1e3451ab7fdde33556a595c (127.11.223.131:33065)
I20260621 08:00:24.242545 29344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ece9ef732b41fcbbb35093fed34625" candidate_uuid: "421fbbeafd234188975210af11eee247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db80b4fe93546e68b7fa02efe9e52db"
I20260621 08:00:24.242558 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ece9ef732b41fcbbb35093fed34625" candidate_uuid: "421fbbeafd234188975210af11eee247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e5293bc1e3451ab7fdde33556a595c"
I20260621 08:00:24.242666 29344 raft_consensus.cc:3037] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:24.242666 29612 raft_consensus.cc:3037] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:24.243899 29344 raft_consensus.cc:2454] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 421fbbeafd234188975210af11eee247 in term 1.
I20260621 08:00:24.243899 29612 raft_consensus.cc:2454] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 421fbbeafd234188975210af11eee247 in term 1.
I20260621 08:00:24.244171 29411 leader_election.cc:302] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 421fbbeafd234188975210af11eee247, 7db80b4fe93546e68b7fa02efe9e52db; no voters:
I20260621 08:00:24.244349 29683 raft_consensus.cc:2781] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:24.244552 29683 raft_consensus.cc:686] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 1 LEADER]: Becoming Leader. State: Replica: 421fbbeafd234188975210af11eee247, State: Running, Role: LEADER
I20260621 08:00:24.244892 29683 consensus_queue.cc:237] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } }
I20260621 08:00:24.247807 29198 catalog_manager.cc:5495] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 reported cstate change: term changed from 0 to 1, leader changed from <none> to 421fbbeafd234188975210af11eee247 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "421fbbeafd234188975210af11eee247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421fbbeafd234188975210af11eee247" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260621 08:00:24.310940 29670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.307626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.11.223.130:39857)
0621 08:00:24.307762 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:24.307766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:24.307779 (+ 13us) server_negotiation.cc:407] Connection header received
0621 08:00:24.307863 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:24.307865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:24.307920 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:24.308005 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:00:24.308215 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.308729 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.309556 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.309816 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.309894 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.310137 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:00:24.310565 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:00:24.310653 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:24.310721 (+ 68us) server_negotiation.cc:299] Negotiation successful
0621 08:00:24.310766 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 08:00:24.316416 29612 raft_consensus.cc:1261] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 1 FOLLOWER]: Refusing update from remote peer 421fbbeafd234188975210af11eee247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:24.316442 29344 raft_consensus.cc:1261] T 71ece9ef732b41fcbbb35093fed34625 P 7db80b4fe93546e68b7fa02efe9e52db [term 1 FOLLOWER]: Refusing update from remote peer 421fbbeafd234188975210af11eee247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:24.317118 29686 consensus_queue.cc:1035] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e5293bc1e3451ab7fdde33556a595c" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:24.317284 29683 consensus_queue.cc:1035] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7db80b4fe93546e68b7fa02efe9e52db" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:24.331872 29198 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:24.332008 29198 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:24.334231 29198 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebc14eda049a4dbabf5ce5ebbd1ac06e: Sending DeleteTablet for 3 replicas of tablet 71ece9ef732b41fcbbb35093fed34625
I20260621 08:00:24.334911 29458 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ece9ef732b41fcbbb35093fed34625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:39724
I20260621 08:00:24.335031 29324 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ece9ef732b41fcbbb35093fed34625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:50088
I20260621 08:00:24.335453 29699 tablet_replica.cc:330] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: stopping tablet replica
I20260621 08:00:24.335484 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29261
I20260621 08:00:24.335564 29592 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ece9ef732b41fcbbb35093fed34625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:24 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35956
I20260621 08:00:24.335745 29699 raft_consensus.cc:2229] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 1 LEADER]: Raft consensus shutting down.
I20260621 08:00:24.336015 29699 raft_consensus.cc:2258] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:24.337114 29699 ts_tablet_manager.cc:1905] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:24.339233 29699 ts_tablet_manager.cc:1918] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:24.339342 29699 log.cc:1196] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/71ece9ef732b41fcbbb35093fed34625
I20260621 08:00:24.339643 29699 ts_tablet_manager.cc:1939] T 71ece9ef732b41fcbbb35093fed34625 P 421fbbeafd234188975210af11eee247: Deleting consensus metadata
I20260621 08:00:24.340512 29701 tablet_replica.cc:330] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: stopping tablet replica
I20260621 08:00:24.340780 29183 catalog_manager.cc:4841] TS 421fbbeafd234188975210af11eee247 (127.11.223.130:39857): tablet 71ece9ef732b41fcbbb35093fed34625 (table test-table [id=961d4a66571f4afdb13292c67ceb7b68]) successfully deleted
I20260621 08:00:24.340854 29701 raft_consensus.cc:2229] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:24.341256 29701 raft_consensus.cc:2258] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:24.342844 29701 ts_tablet_manager.cc:1905] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260621 08:00:24.344008 29183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:39463 (error 108)
I20260621 08:00:24.345017 29671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.344731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39463 (local address 127.0.0.1:50094)
0621 08:00:24.344848 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:24.344922 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260621 08:00:24.345094 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29395
I20260621 08:00:24.345397 29701 ts_tablet_manager.cc:1918] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:24.345484 29701 log.cc:1196] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/71ece9ef732b41fcbbb35093fed34625
I20260621 08:00:24.345904 29701 ts_tablet_manager.cc:1939] T 71ece9ef732b41fcbbb35093fed34625 P 66e5293bc1e3451ab7fdde33556a595c: Deleting consensus metadata
W20260621 08:00:24.346572 29183 catalog_manager.cc:4584] TS 7db80b4fe93546e68b7fa02efe9e52db (127.11.223.129:39463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71ece9ef732b41fcbbb35093fed34625: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39463: connect: Connection refused (error 111)
I20260621 08:00:24.346988 29183 catalog_manager.cc:4841] TS 66e5293bc1e3451ab7fdde33556a595c (127.11.223.131:33065): tablet 71ece9ef732b41fcbbb35093fed34625 (table test-table [id=961d4a66571f4afdb13292c67ceb7b68]) successfully deleted
I20260621 08:00:24.351166 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29529
I20260621 08:00:24.356173 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29167
2026-06-21T08:00:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2475 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-21T08:00:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:24Z Disabled control of system clock
I20260621 08:00:24.384974 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:36775
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:43037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:36775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 08:00:24.490715 29706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:24.490921 29706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:24.490967 29706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:24.494649 29706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:24.494691 29706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:24.494706 29706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:24.494722 29706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:24.494737 29706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:24.499167 29706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:36775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:36775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:24.499987 29706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:24.501173 29706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:24.508975 29714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:24.509060 29706 server_base.cc:1034] running on GCE node
W20260621 08:00:24.508952 29711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:24.508972 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:24.509727 29706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:24.510708 29706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:24.511900 29706 hybrid_clock.cc:648] HybridClock initialized: now 1782028824511869 us; error 41 us; skew 500 ppm
I20260621 08:00:24.513617 29706 webserver.cc:460] Webserver started at http://127.11.223.190:39939/ using document root <none> and password file <none>
I20260621 08:00:24.514146 29706 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:24.514232 29706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:24.514444 29706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:24.516356 29706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "46e65ddd1c104dbc8662bd2c93714561"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "90687bcd0c24418a075949e733d23cfc"
server_key_iv: "149b6f52a4253313a066233e72302c0e"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.516904 29706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "46e65ddd1c104dbc8662bd2c93714561"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "90687bcd0c24418a075949e733d23cfc"
server_key_iv: "149b6f52a4253313a066233e72302c0e"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.520300 29706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 08:00:24.522673 29720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.523846 29706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 08:00:24.523973 29706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "46e65ddd1c104dbc8662bd2c93714561"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "90687bcd0c24418a075949e733d23cfc"
server_key_iv: "149b6f52a4253313a066233e72302c0e"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.524073 29706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:24.540729 29706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:24.541337 29706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:24.541514 29706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:24.548211 29706 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:36775
I20260621 08:00:24.548228 29772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:36775 every 8 connection(s)
I20260621 08:00:24.549075 29706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:24.549784 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29706
I20260621 08:00:24.549918 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:24.550166 12158 external_mini_cluster.cc:1428] Setting key ba4251e7260e6ba02d7363cd19f816d6
I20260621 08:00:24.552420 29773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:24.558755 29773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561: Bootstrap starting.
I20260621 08:00:24.559900 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.551470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.11.223.190:36775)
0621 08:00:24.552097 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:24.552118 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:24.552157 (+ 39us) server_negotiation.cc:407] Connection header received
0621 08:00:24.553004 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:24.553026 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:24.553477 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:24.553832 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:24.554237 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.555658 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.556556 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.557212 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.557382 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.558163 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:24.558168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:24.558182 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:24.558623 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:24.558686 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:24.558709 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:24.558810 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:24.558997 (+ 187us) server_negotiation.cc:299] Negotiation successful
0621 08:00:24.559261 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
I20260621 08:00:24.561026 29773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:24.561823 29773 log.cc:824] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:24.563494 29773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561: No bootstrap required, opened a new log
I20260621 08:00:24.567091 29773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e65ddd1c104dbc8662bd2c93714561" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36775 } }
I20260621 08:00:24.567312 29773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:24.567394 29773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e65ddd1c104dbc8662bd2c93714561, State: Initialized, Role: FOLLOWER
I20260621 08:00:24.567783 29773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e65ddd1c104dbc8662bd2c93714561" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36775 } }
I20260621 08:00:24.567927 29773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:24.568024 29773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:24.568113 29773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:24.568924 29773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e65ddd1c104dbc8662bd2c93714561" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36775 } }
I20260621 08:00:24.569232 29773 leader_election.cc:302] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46e65ddd1c104dbc8662bd2c93714561; no voters:
I20260621 08:00:24.569511 29773 leader_election.cc:288] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:24.569676 29778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:24.569898 29778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [term 1 LEADER]: Becoming Leader. State: Replica: 46e65ddd1c104dbc8662bd2c93714561, State: Running, Role: LEADER
I20260621 08:00:24.570197 29778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e65ddd1c104dbc8662bd2c93714561" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36775 } }
I20260621 08:00:24.570648 29773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:24.571638 29779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46e65ddd1c104dbc8662bd2c93714561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e65ddd1c104dbc8662bd2c93714561" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36775 } } }
I20260621 08:00:24.571763 29779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:24.572259 29787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:24.572265 29780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46e65ddd1c104dbc8662bd2c93714561. Latest consensus state: current_term: 1 leader_uuid: "46e65ddd1c104dbc8662bd2c93714561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e65ddd1c104dbc8662bd2c93714561" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 36775 } } }
I20260621 08:00:24.572942 29780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:24.575253 29787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:24.580809 29787 catalog_manager.cc:1347] Generated new cluster ID: 07e05acc3bad4be18a0809ae80d4fc66
I20260621 08:00:24.580893 29787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:24.606946 29787 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:24.607836 29787 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:24.618496 29787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561: Generated new TSK 0
I20260621 08:00:24.619123 29787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:24.625074 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36775
--builtin_ntp_servers=127.11.223.148:43037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 08:00:24.729470 29797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:24.729686 29797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:24.729719 29797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:24.733373 29797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:24.733412 29797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:24.733497 29797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:24.737998 29797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:24.738848 29797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:24.740037 29797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:24.748744 29805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:24.748726 29797 server_base.cc:1034] running on GCE node
W20260621 08:00:24.748724 29802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:24.748711 29803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:24.749342 29797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:24.749851 29797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:24.751003 29797 hybrid_clock.cc:648] HybridClock initialized: now 1782028824750986 us; error 29 us; skew 500 ppm
I20260621 08:00:24.752700 29797 webserver.cc:460] Webserver started at http://127.11.223.129:34033/ using document root <none> and password file <none>
I20260621 08:00:24.753207 29797 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:24.753283 29797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:24.753517 29797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:24.755347 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "d31636ec12a247cd8d954064621d8eed"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "6c267f76484aee64b1588c189774b7fc"
server_key_iv: "44e2f793f319f3b065eb86fca7b00a8f"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.755877 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "d31636ec12a247cd8d954064621d8eed"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "6c267f76484aee64b1588c189774b7fc"
server_key_iv: "44e2f793f319f3b065eb86fca7b00a8f"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.759203 29797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260621 08:00:24.761482 29811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.762559 29797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:24.762668 29797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "d31636ec12a247cd8d954064621d8eed"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "6c267f76484aee64b1588c189774b7fc"
server_key_iv: "44e2f793f319f3b065eb86fca7b00a8f"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.762766 29797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:24.780221 29797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:24.780826 29797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:24.781006 29797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:24.781483 29797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:24.782294 29797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:24.782336 29797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.782401 29797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:24.782433 29797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.790346 29797 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:41311
I20260621 08:00:24.790376 29924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:41311 every 8 connection(s)
I20260621 08:00:24.791105 29797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 08:00:24.799131 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.792742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:60901 (local address 127.11.223.190:36775)
0621 08:00:24.792889 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:24.792894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:24.793817 (+ 923us) server_negotiation.cc:407] Connection header received
0621 08:00:24.795060 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:24.795081 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:24.795123 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:24.795221 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:24.795972 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.796502 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.797590 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.797787 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.797860 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.798765 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:24.798768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:24.798770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:24.798779 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:24.798797 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:24.798803 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:24.798855 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:24.798974 (+ 119us) server_negotiation.cc:299] Negotiation successful
0621 08:00:24.799031 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 08:00:24.799839 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.793064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:36775 (local address 127.11.223.129:60901)
0621 08:00:24.793642 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:24.793678 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:24.794873 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:24.795366 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:24.795371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:24.795445 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:24.795859 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.795870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.796601 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.796603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.797481 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.797489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.797906 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.797909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.798031 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.798553 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:24.798554 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:24.798589 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:24.798641 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:24.798868 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:24.798874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:24.798877 (+ 3us) client_negotiation.cc:769] Sending connection context
0621 08:00:24.798955 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 08:00:24.799131 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":96,"threads_started":1}
I20260621 08:00:24.800819 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29797
I20260621 08:00:24.800978 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 08:00:24.801004 29925 heartbeater.cc:347] Connected to a master server at 127.11.223.190:36775
I20260621 08:00:24.801244 29925 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:24.801316 12158 external_mini_cluster.cc:1428] Setting key 460c555c6260c44e9b72a632bd5e9dd6
I20260621 08:00:24.801771 29925 heartbeater.cc:511] Master 127.11.223.190:36775 requested a full tablet report, sending...
I20260621 08:00:24.803251 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36775
--builtin_ntp_servers=127.11.223.148:43037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 08:00:24.803306 29737 ts_manager.cc:194] Registered new tserver with Master: d31636ec12a247cd8d954064621d8eed (127.11.223.129:41311)
I20260621 08:00:24.804886 29737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.129:60901
W20260621 08:00:24.906983 29928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:24.907262 29928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:24.907325 29928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:24.911128 29928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:24.911216 29928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:24.911302 29928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:24.915727 29928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:24.916631 29928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:24.917860 29928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:24.926815 29934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:24.926853 29933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:24.927030 29928 server_base.cc:1034] running on GCE node
W20260621 08:00:24.926822 29936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:24.927520 29928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:24.928188 29928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:24.929338 29928 hybrid_clock.cc:648] HybridClock initialized: now 1782028824929311 us; error 40 us; skew 500 ppm
I20260621 08:00:24.930863 29928 webserver.cc:460] Webserver started at http://127.11.223.130:38295/ using document root <none> and password file <none>
I20260621 08:00:24.931340 29928 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:24.931396 29928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:24.931542 29928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:24.933321 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "2d697030a09743ce9b2b0894bc81e46c"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "2fb89e1c3c5db6de3b444f7ed7d4f125"
server_key_iv: "f3dc7b2a158b284fe2f0a1ede21cf599"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.933856 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "2d697030a09743ce9b2b0894bc81e46c"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "2fb89e1c3c5db6de3b444f7ed7d4f125"
server_key_iv: "f3dc7b2a158b284fe2f0a1ede21cf599"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.937136 29928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:24.939328 29942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.940476 29928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:24.940593 29928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "2d697030a09743ce9b2b0894bc81e46c"
format_stamp: "Formatted at 2026-06-21 08:00:24 on dist-test-slave-0pn3"
server_key: "2fb89e1c3c5db6de3b444f7ed7d4f125"
server_key_iv: "f3dc7b2a158b284fe2f0a1ede21cf599"
server_key_version: "encryptionkey@0"
I20260621 08:00:24.940698 29928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:24.962095 29928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:24.962687 29928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:24.962860 29928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:24.963367 29928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:24.964231 29928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:24.964300 29928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.964380 29928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:24.964424 29928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:24.973341 29928 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:33099
I20260621 08:00:24.973364 30055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:33099 every 8 connection(s)
I20260621 08:00:24.974182 29928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 08:00:24.979880 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 29928
I20260621 08:00:24.980005 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 08:00:24.980285 12158 external_mini_cluster.cc:1428] Setting key 0592b43616779cf4116e6554fdfedb0f
I20260621 08:00:24.982441 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36775
--builtin_ntp_servers=127.11.223.148:43037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 08:00:24.983846 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.975950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:51671 (local address 127.11.223.190:36775)
0621 08:00:24.976093 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:24.976097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:24.977146 (+ 1049us) server_negotiation.cc:407] Connection header received
0621 08:00:24.978627 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:24.978631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:24.978690 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:24.978782 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:24.979647 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.980298 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.981822 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.982137 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.982258 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.983273 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:24.983279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:24.983285 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:24.983301 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:24.983327 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:24.983339 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:24.983419 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:24.983545 (+ 126us) server_negotiation.cc:299] Negotiation successful
0621 08:00:24.983618 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260621 08:00:24.984786 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:24.976354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:36775 (local address 127.11.223.130:51671)
0621 08:00:24.976982 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:24.977039 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:24.978420 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:24.978961 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:24.978968 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:24.979047 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:24.979499 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.979511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.980442 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.980448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.981695 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:24.981705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:24.982282 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:24.982284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:24.982413 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:24.983030 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:24.983032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:24.983067 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:24.983122 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:24.983423 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:24.983428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:24.983432 (+ 4us) client_negotiation.cc:769] Sending connection context
0621 08:00:24.983519 (+ 87us) client_negotiation.cc:240] Negotiation successful
0621 08:00:24.983714 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":100,"threads_started":1}
I20260621 08:00:24.985926 30056 heartbeater.cc:347] Connected to a master server at 127.11.223.190:36775
I20260621 08:00:24.986157 30056 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:24.986634 30056 heartbeater.cc:511] Master 127.11.223.190:36775 requested a full tablet report, sending...
I20260621 08:00:24.987656 29737 ts_manager.cc:194] Registered new tserver with Master: 2d697030a09743ce9b2b0894bc81e46c (127.11.223.130:33099)
I20260621 08:00:24.988564 29737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.130:51671
W20260621 08:00:25.089798 30059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:25.090001 30059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:25.090034 30059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:25.093863 30059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:25.093906 30059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:25.093986 30059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:25.100878 30059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:36775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:25.102222 30059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:25.103855 30059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:25.114199 30067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:25.114213 30064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:25.114363 30065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:25.114631 30059 server_base.cc:1034] running on GCE node
I20260621 08:00:25.114987 30059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:25.115583 30059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:25.116757 30059 hybrid_clock.cc:648] HybridClock initialized: now 1782028825116744 us; error 36 us; skew 500 ppm
I20260621 08:00:25.118466 30059 webserver.cc:460] Webserver started at http://127.11.223.131:34781/ using document root <none> and password file <none>
I20260621 08:00:25.119042 30059 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:25.119130 30059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:25.119369 30059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:25.121346 30059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "bacaab4b6cfa4b289c9aa113116258f9"
format_stamp: "Formatted at 2026-06-21 08:00:25 on dist-test-slave-0pn3"
server_key: "54fbd02c563ec09fc988af102518c750"
server_key_iv: "9a7e11d5b038a4bdd543184e6b13e9a0"
server_key_version: "encryptionkey@0"
I20260621 08:00:25.121863 30059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "bacaab4b6cfa4b289c9aa113116258f9"
format_stamp: "Formatted at 2026-06-21 08:00:25 on dist-test-slave-0pn3"
server_key: "54fbd02c563ec09fc988af102518c750"
server_key_iv: "9a7e11d5b038a4bdd543184e6b13e9a0"
server_key_version: "encryptionkey@0"
I20260621 08:00:25.125219 30059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:25.127347 30073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:25.128553 30059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:25.128670 30059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "bacaab4b6cfa4b289c9aa113116258f9"
format_stamp: "Formatted at 2026-06-21 08:00:25 on dist-test-slave-0pn3"
server_key: "54fbd02c563ec09fc988af102518c750"
server_key_iv: "9a7e11d5b038a4bdd543184e6b13e9a0"
server_key_version: "encryptionkey@0"
I20260621 08:00:25.128767 30059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:25.149607 30059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:25.150193 30059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:25.150381 30059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:25.150919 30059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:25.151772 30059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:25.151841 30059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:25.151912 30059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:25.151964 30059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:25.160373 30059 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:37093
I20260621 08:00:25.160413 30186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:37093 every 8 connection(s)
I20260621 08:00:25.161175 30059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:25.170622 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30059
I20260621 08:00:25.170739 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 08:00:25.171011 12158 external_mini_cluster.cc:1428] Setting key 7ed1fa067c14eab5e3a2853a0f32ed7a
I20260621 08:00:25.171012 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.163132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:55891 (local address 127.11.223.190:36775)
0621 08:00:25.163293 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.163298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.164573 (+ 1275us) server_negotiation.cc:407] Connection header received
0621 08:00:25.166440 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.166443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.166499 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.166605 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:25.167608 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.168106 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.169190 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.169387 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.169464 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.170486 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:25.170489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:25.170492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:25.170508 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:25.170535 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:25.170546 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:25.170629 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.170815 (+ 186us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.170873 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260621 08:00:25.172003 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.163557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:36775 (local address 127.11.223.131:55891)
0621 08:00:25.164417 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.164468 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:25.166188 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:25.166795 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:25.166805 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:25.166914 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:25.167483 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.167495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.168218 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.168220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.169090 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.169099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.169509 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.169511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.169635 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.170202 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:25.170213 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:25.170264 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:25.170332 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:25.170635 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:25.170641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:25.170645 (+ 4us) client_negotiation.cc:769] Sending connection context
0621 08:00:25.170771 (+ 126us) client_negotiation.cc:240] Negotiation successful
0621 08:00:25.171018 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260621 08:00:25.173074 30187 heartbeater.cc:347] Connected to a master server at 127.11.223.190:36775
I20260621 08:00:25.173281 30187 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:25.173702 30187 heartbeater.cc:511] Master 127.11.223.190:36775 requested a full tablet report, sending...
I20260621 08:00:25.174573 29737 ts_manager.cc:194] Registered new tserver with Master: bacaab4b6cfa4b289c9aa113116258f9 (127.11.223.131:37093)
I20260621 08:00:25.175100 29737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.131:55891
I20260621 08:00:25.184799 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 08:00:25.190105 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.186664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.11.223.190:36775)
0621 08:00:25.186803 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.186807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.186979 (+ 172us) server_negotiation.cc:407] Connection header received
0621 08:00:25.187061 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.187064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.187100 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.187156 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:25.187375 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.187834 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.188697 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.188912 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.188983 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.189751 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:25.189753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:25.189755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:25.189765 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:25.189780 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:25.189787 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:25.189838 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.189900 (+ 62us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.189939 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 08:00:25.192857 29737 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 08:00:25.195629 29737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 08:00:25.209261 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.203574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33099 (local address 127.0.0.1:35038)
0621 08:00:25.204003 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.204016 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:25.204085 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:25.205322 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:25.205328 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:25.205350 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:25.205514 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.205525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.206553 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.206559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.207731 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.207740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.208725 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.208729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.208837 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.208872 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 08:00:25.208989 (+ 117us) client_negotiation.cc:240] Negotiation successful
0621 08:00:25.209104 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260621 08:00:25.209266 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.204451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.11.223.130:33099)
0621 08:00:25.204817 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.204829 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.204854 (+ 25us) server_negotiation.cc:407] Connection header received
0621 08:00:25.204935 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.204941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.205165 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.205333 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:25.205647 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.206442 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.207836 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.208585 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.208743 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.208918 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.209016 (+ 98us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.209106 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":112,"threads_started":1}
I20260621 08:00:25.209502 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.204158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:41311 (local address 127.0.0.1:44600)
0621 08:00:25.204452 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.204467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:25.204558 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:25.205400 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:25.205403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:25.205421 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:25.205643 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.205652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.207039 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.207043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.208029 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.208036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.209127 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.209131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.209213 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.209235 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 08:00:25.209286 (+ 51us) client_negotiation.cc:240] Negotiation successful
0621 08:00:25.209355 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":70,"thread_start_us":92,"threads_started":1}
I20260621 08:00:25.209730 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.203467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:37093 (local address 127.0.0.1:42664)
0621 08:00:25.203814 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.203863 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:25.203986 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:25.205490 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:25.205493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:25.205507 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:25.205667 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.205673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.207078 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.207081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.208160 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.208170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.209347 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.209350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.209428 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.209447 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 08:00:25.209525 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 08:00:25.209602 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":31,"thread_start_us":75,"threads_started":1}
I20260621 08:00:25.209892 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.204148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.11.223.131:37093)
0621 08:00:25.204999 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.205009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.205028 (+ 19us) server_negotiation.cc:407] Connection header received
0621 08:00:25.205104 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.205108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.205255 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.205391 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:25.205785 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.206920 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.208292 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.209007 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.209122 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.209291 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.209578 (+ 287us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.209681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":577,"threads_started":1}
I20260621 08:00:25.210256 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.204392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.11.223.129:41311)
0621 08:00:25.204910 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.204921 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.204945 (+ 24us) server_negotiation.cc:407] Connection header received
0621 08:00:25.205022 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.205027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.205236 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.205616 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:25.205770 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.206920 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.208136 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.208985 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.209805 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.209993 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.210042 (+ 49us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.210136 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":120,"threads_started":1}
I20260621 08:00:25.212648 29859 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 (DEFAULT_TABLE table=test-table [id=37122dd9c7804449995308ee9826b8d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:25.212653 29990 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 (DEFAULT_TABLE table=test-table [id=37122dd9c7804449995308ee9826b8d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:25.212821 30121 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 (DEFAULT_TABLE table=test-table [id=37122dd9c7804449995308ee9826b8d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:25.213683 30121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:25.213713 29990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:25.213713 29859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:25.219709 30204 tablet_bootstrap.cc:492] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Bootstrap starting.
I20260621 08:00:25.220949 30206 tablet_bootstrap.cc:492] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Bootstrap starting.
I20260621 08:00:25.221334 30205 tablet_bootstrap.cc:492] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: Bootstrap starting.
I20260621 08:00:25.222086 30204 tablet_bootstrap.cc:654] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:25.222770 30204 log.cc:824] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:25.223001 30205 tablet_bootstrap.cc:654] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:25.223637 30205 log.cc:824] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:25.224598 30204 tablet_bootstrap.cc:492] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: No bootstrap required, opened a new log
I20260621 08:00:25.224785 30204 ts_tablet_manager.cc:1397] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 08:00:25.225826 30206 tablet_bootstrap.cc:654] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:25.226429 30206 log.cc:824] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:25.227918 30205 tablet_bootstrap.cc:492] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: No bootstrap required, opened a new log
I20260621 08:00:25.228109 30205 ts_tablet_manager.cc:1397] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260621 08:00:25.228379 30206 tablet_bootstrap.cc:492] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: No bootstrap required, opened a new log
I20260621 08:00:25.228549 30206 ts_tablet_manager.cc:1397] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260621 08:00:25.228434 30204 raft_consensus.cc:348] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.228691 30204 raft_consensus.cc:374] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:25.228760 30204 raft_consensus.cc:729] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacaab4b6cfa4b289c9aa113116258f9, State: Initialized, Role: FOLLOWER
I20260621 08:00:25.229254 30204 consensus_queue.cc:260] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.229892 30187 heartbeater.cc:503] Master 127.11.223.190:36775 was elected leader, sending a full tablet report...
I20260621 08:00:25.230240 30204 ts_tablet_manager.cc:1428] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 08:00:25.232350 30206 raft_consensus.cc:348] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.232673 30206 raft_consensus.cc:374] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:25.232784 30206 raft_consensus.cc:729] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d697030a09743ce9b2b0894bc81e46c, State: Initialized, Role: FOLLOWER
I20260621 08:00:25.232962 30205 raft_consensus.cc:348] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.233374 30205 raft_consensus.cc:374] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:25.233493 30205 raft_consensus.cc:729] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31636ec12a247cd8d954064621d8eed, State: Initialized, Role: FOLLOWER
I20260621 08:00:25.234181 30205 consensus_queue.cc:260] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.235569 30205 ts_tablet_manager.cc:1428] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260621 08:00:25.236037 29925 heartbeater.cc:503] Master 127.11.223.190:36775 was elected leader, sending a full tablet report...
I20260621 08:00:25.236053 30206 consensus_queue.cc:260] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.237058 30206 ts_tablet_manager.cc:1428] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260621 08:00:25.238046 30056 heartbeater.cc:503] Master 127.11.223.190:36775 was elected leader, sending a full tablet report...
I20260621 08:00:25.245918 30211 raft_consensus.cc:482] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:25.246054 30211 raft_consensus.cc:504] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.247130 30211 leader_election.cc:288] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bacaab4b6cfa4b289c9aa113116258f9 (127.11.223.131:37093), 2d697030a09743ce9b2b0894bc81e46c (127.11.223.130:33099)
I20260621 08:00:25.251819 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.247443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:54969 (local address 127.11.223.131:37093)
0621 08:00:25.247575 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.247580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.247597 (+ 17us) server_negotiation.cc:407] Connection header received
0621 08:00:25.247702 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.247730 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.247825 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.247892 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:25.248200 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.249040 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.250574 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.251155 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.251290 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.251337 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.251614 (+ 277us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.251689 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 08:00:25.251819 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.247340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:37093 (local address 127.11.223.129:54969)
0621 08:00:25.247495 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.247511 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:25.247602 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:25.247927 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:25.247931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:25.247948 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:25.248087 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.248095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.249152 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.249155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.250426 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.250437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.251374 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.251377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.251485 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.251504 (+ 19us) client_negotiation.cc:769] Sending connection context
0621 08:00:25.251586 (+ 82us) client_negotiation.cc:240] Negotiation successful
0621 08:00:25.251659 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1}
I20260621 08:00:25.251855 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.247494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:49433 (local address 127.11.223.130:33099)
0621 08:00:25.247593 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.247597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.248024 (+ 427us) server_negotiation.cc:407] Connection header received
0621 08:00:25.248161 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.248165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.248214 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.248290 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:25.248564 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.249243 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.250582 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.251155 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.251290 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.251337 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.251661 (+ 324us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.251739 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260621 08:00:25.251880 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.247340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:33099 (local address 127.11.223.129:49433)
0621 08:00:25.247947 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.247961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:25.248052 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:25.248305 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:25.248307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:25.248362 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:25.248463 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.248467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.249329 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.249332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.250461 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:25.250470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.251432 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.251435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:25.251541 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.251557 (+ 16us) client_negotiation.cc:769] Sending connection context
0621 08:00:25.251645 (+ 88us) client_negotiation.cc:240] Negotiation successful
0621 08:00:25.251714 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":212,"thread_start_us":114,"threads_started":1}
I20260621 08:00:25.252429 30010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad23dfdf8cc478eb6f83e4cf4e60dd1" candidate_uuid: "d31636ec12a247cd8d954064621d8eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d697030a09743ce9b2b0894bc81e46c" is_pre_election: true
I20260621 08:00:25.252746 30010 raft_consensus.cc:2454] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d31636ec12a247cd8d954064621d8eed in term 0.
I20260621 08:00:25.252812 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad23dfdf8cc478eb6f83e4cf4e60dd1" candidate_uuid: "d31636ec12a247cd8d954064621d8eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bacaab4b6cfa4b289c9aa113116258f9" is_pre_election: true
I20260621 08:00:25.253057 30141 raft_consensus.cc:2454] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d31636ec12a247cd8d954064621d8eed in term 0.
I20260621 08:00:25.253288 29815 leader_election.cc:302] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d697030a09743ce9b2b0894bc81e46c, d31636ec12a247cd8d954064621d8eed; no voters:
I20260621 08:00:25.253571 30211 raft_consensus.cc:2781] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 08:00:25.253669 30211 raft_consensus.cc:482] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:25.253724 30211 raft_consensus.cc:3037] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:25.254706 30211 raft_consensus.cc:504] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.255048 30211 leader_election.cc:288] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [CANDIDATE]: Term 1 election: Requested vote from peers bacaab4b6cfa4b289c9aa113116258f9 (127.11.223.131:37093), 2d697030a09743ce9b2b0894bc81e46c (127.11.223.130:33099)
I20260621 08:00:25.255370 30010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad23dfdf8cc478eb6f83e4cf4e60dd1" candidate_uuid: "d31636ec12a247cd8d954064621d8eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d697030a09743ce9b2b0894bc81e46c"
I20260621 08:00:25.255370 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad23dfdf8cc478eb6f83e4cf4e60dd1" candidate_uuid: "d31636ec12a247cd8d954064621d8eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bacaab4b6cfa4b289c9aa113116258f9"
I20260621 08:00:25.255488 30010 raft_consensus.cc:3037] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:25.255496 30141 raft_consensus.cc:3037] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:25.256778 30010 raft_consensus.cc:2454] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31636ec12a247cd8d954064621d8eed in term 1.
I20260621 08:00:25.256778 30141 raft_consensus.cc:2454] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31636ec12a247cd8d954064621d8eed in term 1.
I20260621 08:00:25.257077 29815 leader_election.cc:302] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d697030a09743ce9b2b0894bc81e46c, bacaab4b6cfa4b289c9aa113116258f9, d31636ec12a247cd8d954064621d8eed; no voters:
I20260621 08:00:25.257275 30211 raft_consensus.cc:2781] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:25.257491 30211 raft_consensus.cc:686] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 1 LEADER]: Becoming Leader. State: Replica: d31636ec12a247cd8d954064621d8eed, State: Running, Role: LEADER
I20260621 08:00:25.257819 30211 consensus_queue.cc:237] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } }
I20260621 08:00:25.260865 29736 catalog_manager.cc:5495] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed reported cstate change: term changed from 0 to 1, leader changed from <none> to d31636ec12a247cd8d954064621d8eed (127.11.223.129). New cstate: current_term: 1 leader_uuid: "d31636ec12a247cd8d954064621d8eed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d31636ec12a247cd8d954064621d8eed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 41311 } health_report { overall_health: HEALTHY } } }
I20260621 08:00:25.271454 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.268108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.11.223.129:41311)
0621 08:00:25.268257 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:25.268262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:25.268304 (+ 42us) server_negotiation.cc:407] Connection header received
0621 08:00:25.268529 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:25.268532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:25.268593 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:25.268671 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:00:25.268883 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.269366 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.270127 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:25.270356 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:25.270500 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:25.270786 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:00:25.271135 (+ 349us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:00:25.271228 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:25.271285 (+ 57us) server_negotiation.cc:299] Negotiation successful
0621 08:00:25.271325 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260621 08:00:25.276700 30010 raft_consensus.cc:1261] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 1 FOLLOWER]: Refusing update from remote peer d31636ec12a247cd8d954064621d8eed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:25.276715 30141 raft_consensus.cc:1261] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 1 FOLLOWER]: Refusing update from remote peer d31636ec12a247cd8d954064621d8eed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:25.277318 30214 consensus_queue.cc:1035] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [LEADER]: Connected to new peer: Peer: permanent_uuid: "bacaab4b6cfa4b289c9aa113116258f9" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 37093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:25.277477 30211 consensus_queue.cc:1035] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d697030a09743ce9b2b0894bc81e46c" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:25.283881 30217 mvcc.cc:205] Tried to move back new op lower bound from 7299190068326985728 to 7299190068258930688. Current Snapshot: MvccSnapshot[applied={T|T < 7299190068326985728}]
I20260621 08:00:25.284587 30218 mvcc.cc:205] Tried to move back new op lower bound from 7299190068326985728 to 7299190068258930688. Current Snapshot: MvccSnapshot[applied={T|T < 7299190068326985728}]
I20260621 08:00:25.287439 30219 mvcc.cc:205] Tried to move back new op lower bound from 7299190068326985728 to 7299190068258930688. Current Snapshot: MvccSnapshot[applied={T|T < 7299190068326985728}]
I20260621 08:00:25.292266 29736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:25.292467 29736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:25.294767 29736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46e65ddd1c104dbc8662bd2c93714561: Sending DeleteTablet for 3 replicas of tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1
I20260621 08:00:25.295469 30121 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:25 UTC) from {username='slave'} at 127.0.0.1:42664
I20260621 08:00:25.295671 29990 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:25 UTC) from {username='slave'} at 127.0.0.1:35038
I20260621 08:00:25.295763 29859 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:25 UTC) from {username='slave'} at 127.0.0.1:44600
I20260621 08:00:25.296183 30229 tablet_replica.cc:330] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed: stopping tablet replica
I20260621 08:00:25.296489 30229 raft_consensus.cc:2229] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P d31636ec12a247cd8d954064621d8eed [term 1 LEADER]: Raft consensus shutting down.
I20260621 08:00:25.296573 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29797
I20260621 08:00:25.297241 30228 tablet_replica.cc:330] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: stopping tablet replica
I20260621 08:00:25.297384 30227 tablet_replica.cc:330] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: stopping tablet replica
I20260621 08:00:25.297529 30228 raft_consensus.cc:2229] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:25.297617 30227 raft_consensus.cc:2229] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:25.297824 30228 raft_consensus.cc:2258] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:25.297823 30227 raft_consensus.cc:2258] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:25.299376 30228 ts_tablet_manager.cc:1905] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:25.299451 30227 ts_tablet_manager.cc:1905] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:25.302069 30228 ts_tablet_manager.cc:1918] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:25.302069 30227 ts_tablet_manager.cc:1918] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:25.302170 30228 log.cc:1196] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/7ad23dfdf8cc478eb6f83e4cf4e60dd1
I20260621 08:00:25.302170 30227 log.cc:1196] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/7ad23dfdf8cc478eb6f83e4cf4e60dd1
I20260621 08:00:25.302480 30228 ts_tablet_manager.cc:1939] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P 2d697030a09743ce9b2b0894bc81e46c: Deleting consensus metadata
I20260621 08:00:25.302480 30227 ts_tablet_manager.cc:1939] T 7ad23dfdf8cc478eb6f83e4cf4e60dd1 P bacaab4b6cfa4b289c9aa113116258f9: Deleting consensus metadata
I20260621 08:00:25.303618 29724 catalog_manager.cc:4841] TS 2d697030a09743ce9b2b0894bc81e46c (127.11.223.130:33099): tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 (table test-table [id=37122dd9c7804449995308ee9826b8d9]) successfully deleted
I20260621 08:00:25.303633 29723 catalog_manager.cc:4841] TS bacaab4b6cfa4b289c9aa113116258f9 (127.11.223.131:37093): tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1 (table test-table [id=37122dd9c7804449995308ee9826b8d9]) successfully deleted
W20260621 08:00:25.303920 29724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:41311 (error 108)
I20260621 08:00:25.304086 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29928
I20260621 08:00:25.305122 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:25.304744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:41311 (local address 127.0.0.1:44626)
0621 08:00:25.304930 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:25.305002 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:41311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260621 08:00:25.305290 29724 catalog_manager.cc:4584] TS d31636ec12a247cd8d954064621d8eed (127.11.223.129:41311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ad23dfdf8cc478eb6f83e4cf4e60dd1: Network error: Client connection negotiation failed: client connection to 127.11.223.129:41311: connect: Connection refused (error 111)
I20260621 08:00:25.309881 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30059
I20260621 08:00:25.315178 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 29706
2026-06-21T08:00:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:25 dist-test-slave-0pn3 krb5kdc[30233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:25 dist-test-slave-0pn3 krb5kdc[30233](info): set up 2 sockets
Jun 21 08:00:25 dist-test-slave-0pn3 krb5kdc[30233](info): commencing operation
krb5kdc: starting...
W20260621 08:00:26.336215 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:26 dist-test-slave-0pn3 krb5kdc[30233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:26.449237 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44467
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:44467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:26.553400 30246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:26.553617 30246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:26.553654 30246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:26.557519 30246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:26.557562 30246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:26.557579 30246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:26.557595 30246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:26.557607 30246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:26.562389 30246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:44467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:26.563288 30246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:26.564217 30246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:26.572188 30252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:26.572206 30251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:26.572188 30254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:26.572507 30246 server_base.cc:1034] running on GCE node
I20260621 08:00:26.573082 30246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:26.574051 30246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:26.575265 30246 hybrid_clock.cc:648] HybridClock initialized: now 1782028826575221 us; error 34 us; skew 500 ppm
Jun 21 08:00:26 dist-test-slave-0pn3 krb5kdc[30233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028826, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:26.582376 30246 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:26.583134 30246 webserver.cc:460] Webserver started at http://127.11.223.190:35371/ using document root <none> and password file <none>
I20260621 08:00:26.583640 30246 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:26.583725 30246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:26.583953 30246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:26.585883 30246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "60df14d7616347b298e90d7f30b0cccf"
format_stamp: "Formatted at 2026-06-21 08:00:26 on dist-test-slave-0pn3"
server_key: "25ef52d5ff56a078112ffee6b881fe0a"
server_key_iv: "71e0d3bb12ffa9cee57b3c701a45ae9e"
server_key_version: "encryptionkey@0"
I20260621 08:00:26.586416 30246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "60df14d7616347b298e90d7f30b0cccf"
format_stamp: "Formatted at 2026-06-21 08:00:26 on dist-test-slave-0pn3"
server_key: "25ef52d5ff56a078112ffee6b881fe0a"
server_key_iv: "71e0d3bb12ffa9cee57b3c701a45ae9e"
server_key_version: "encryptionkey@0"
I20260621 08:00:26.589844 30246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260621 08:00:26.592029 30261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:26.593241 30246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260621 08:00:26.593375 30246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "60df14d7616347b298e90d7f30b0cccf"
format_stamp: "Formatted at 2026-06-21 08:00:26 on dist-test-slave-0pn3"
server_key: "25ef52d5ff56a078112ffee6b881fe0a"
server_key_iv: "71e0d3bb12ffa9cee57b3c701a45ae9e"
server_key_version: "encryptionkey@0"
I20260621 08:00:26.593482 30246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:26.626607 30246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:26.629303 30246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:26.629495 30246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:26.636231 30246 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:44467
I20260621 08:00:26.636248 30313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:44467 every 8 connection(s)
I20260621 08:00:26.637113 30246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:26.639632 30314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:26.644817 30314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: Bootstrap starting.
I20260621 08:00:26.644820 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30246
I20260621 08:00:26.644990 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:26.645315 12158 external_mini_cluster.cc:1428] Setting key 0fc578ffd57c8a523b05d4cc92abd420
I20260621 08:00:26.647295 30314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:26.647996 30314 log.cc:824] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:26.649961 30314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: No bootstrap required, opened a new log
Jun 21 08:00:26 dist-test-slave-0pn3 krb5kdc[30233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:26.654253 30314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60df14d7616347b298e90d7f30b0cccf" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44467 } }
I20260621 08:00:26.654474 30314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:26.654551 30314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60df14d7616347b298e90d7f30b0cccf, State: Initialized, Role: FOLLOWER
I20260621 08:00:26.654946 30314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60df14d7616347b298e90d7f30b0cccf" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44467 } }
I20260621 08:00:26.655105 30314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:26.655217 30314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:26.655329 30314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:26.656440 30314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60df14d7616347b298e90d7f30b0cccf" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44467 } }
I20260621 08:00:26.656795 30314 leader_election.cc:302] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60df14d7616347b298e90d7f30b0cccf; no voters:
I20260621 08:00:26.657132 30314 leader_election.cc:288] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:26.657191 30319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:26.657392 30319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [term 1 LEADER]: Becoming Leader. State: Replica: 60df14d7616347b298e90d7f30b0cccf, State: Running, Role: LEADER
I20260621 08:00:26.657660 30319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60df14d7616347b298e90d7f30b0cccf" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44467 } }
I20260621 08:00:26.658306 30314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:26.658737 30319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60df14d7616347b298e90d7f30b0cccf. Latest consensus state: current_term: 1 leader_uuid: "60df14d7616347b298e90d7f30b0cccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60df14d7616347b298e90d7f30b0cccf" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44467 } } }
I20260621 08:00:26.658849 30319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:26.659967 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:26.646512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.11.223.190:44467)
0621 08:00:26.646884 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:26.646898 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:26.646929 (+ 31us) server_negotiation.cc:407] Connection header received
0621 08:00:26.647556 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:26.647573 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:26.647890 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:26.648205 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:26.649023 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:26.650065 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:26.651176 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:26.651640 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:26.651759 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:26.654505 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:26.654511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:26.654528 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:26.654579 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:26.656717 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:26.657171 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:26.657176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:26.657182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:26.657343 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:26.657690 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:26.657695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:26.657699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:26.658139 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:26.658346 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:26.658681 (+ 335us) server_negotiation.cc:299] Negotiation successful
0621 08:00:26.658974 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
I20260621 08:00:26.659380 30320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60df14d7616347b298e90d7f30b0cccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60df14d7616347b298e90d7f30b0cccf" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44467 } } }
I20260621 08:00:26.660643 30320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:26.662896 30334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:26.662985 30334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:26.663079 30323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:26.663956 30323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:26.669526 30323 catalog_manager.cc:1347] Generated new cluster ID: fb12d5b36a534a768ac13ab8ef4f06cd
I20260621 08:00:26.669596 30323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:26.683005 30323 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:26.683872 30323 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:26.692188 30323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: Generated new TSK 0
I20260621 08:00:26.692776 30323 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 08:00:26.724921 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44467
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:26.831094 30340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:26.831341 30340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:26.831382 30340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:26.835372 30340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:26.835420 30340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:26.835500 30340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:26.840821 30340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:26.841845 30340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:26.842841 30340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:26.851850 30345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:26.851841 30346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:26.851943 30340 server_base.cc:1034] running on GCE node
W20260621 08:00:26.851845 30348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:26.852430 30340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:26.853022 30340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:26.854231 30340 hybrid_clock.cc:648] HybridClock initialized: now 1782028826854222 us; error 40 us; skew 500 ppm
Jun 21 08:00:26 dist-test-slave-0pn3 krb5kdc[30233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028826, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:26.856890 30340 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 08:00:26.857625 30340 webserver.cc:460] Webserver started at http://127.11.223.129:37749/ using document root <none> and password file <none>
I20260621 08:00:26.858098 30340 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:26.858152 30340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:26.858464 30340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:26.860581 30340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "5613b165c2b34c448782d947e983b60b"
format_stamp: "Formatted at 2026-06-21 08:00:26 on dist-test-slave-0pn3"
server_key: "c86a840f1e50cb1a37567d0ceb6ac44a"
server_key_iv: "3e1ac6593492eb336544ca42730381bc"
server_key_version: "encryptionkey@0"
I20260621 08:00:26.861120 30340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "5613b165c2b34c448782d947e983b60b"
format_stamp: "Formatted at 2026-06-21 08:00:26 on dist-test-slave-0pn3"
server_key: "c86a840f1e50cb1a37567d0ceb6ac44a"
server_key_iv: "3e1ac6593492eb336544ca42730381bc"
server_key_version: "encryptionkey@0"
I20260621 08:00:26.864513 30340 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:26.866712 30355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:26.867830 30340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:26.867957 30340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "5613b165c2b34c448782d947e983b60b"
format_stamp: "Formatted at 2026-06-21 08:00:26 on dist-test-slave-0pn3"
server_key: "c86a840f1e50cb1a37567d0ceb6ac44a"
server_key_iv: "3e1ac6593492eb336544ca42730381bc"
server_key_version: "encryptionkey@0"
I20260621 08:00:26.868067 30340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:26.886549 30340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:26.889559 30340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:26.889750 30340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:26.890264 30340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:26.890986 30340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:26.891053 30340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:26.891114 30340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:26.891165 30340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:26.899709 30340 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:39465
I20260621 08:00:26.899796 30468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:39465 every 8 connection(s)
I20260621 08:00:26.900584 30340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
Jun 21 08:00:26 dist-test-slave-0pn3 krb5kdc[30233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028826, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:26.910358 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30340
I20260621 08:00:26.910491 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 08:00:26.910864 12158 external_mini_cluster.cc:1428] Setting key e240ae25347ae1301d7c5726c140ee60
I20260621 08:00:26.913654 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:26.902257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:58099 (local address 127.11.223.190:44467)
0621 08:00:26.902388 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:26.902392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:26.903132 (+ 740us) server_negotiation.cc:407] Connection header received
0621 08:00:26.904436 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:26.904439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:26.904496 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:26.904606 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:26.905675 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:26.906192 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:26.907319 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:26.907551 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:26.907669 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:26.909898 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:26.909902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:26.909904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:26.909928 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:26.911423 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:26.911936 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:26.911940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:26.911942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:26.912043 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:26.912618 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:26.912622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:26.912624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:26.912795 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:26.912903 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:26.913345 (+ 442us) server_negotiation.cc:299] Negotiation successful
0621 08:00:26.913477 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 08:00:26.914424 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:26.902589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44467 (local address 127.11.223.129:58099)
0621 08:00:26.902996 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:26.903035 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:26.904192 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:26.904768 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:26.904774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:26.905097 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:26.905523 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:26.905541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:26.906312 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:26.906316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:26.907198 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:26.907207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:26.907693 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:26.907698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:26.907863 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:26.908379 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:26.908396 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:26.909753 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:26.911525 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:26.911530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:26.911542 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:26.911820 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:26.912387 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:26.912390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:26.912392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:26.912523 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:26.912900 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:26.912905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:26.913122 (+ 217us) client_negotiation.cc:769] Sending connection context
0621 08:00:26.913288 (+ 166us) client_negotiation.cc:240] Negotiation successful
0621 08:00:26.913598 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":107,"threads_started":1}
I20260621 08:00:26.915637 30469 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44467
I20260621 08:00:26.915866 30469 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:26.916414 30469 heartbeater.cc:511] Master 127.11.223.190:44467 requested a full tablet report, sending...
I20260621 08:00:26.917944 30278 ts_manager.cc:194] Registered new tserver with Master: 5613b165c2b34c448782d947e983b60b (127.11.223.129:39465)
I20260621 08:00:26.919466 30278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:58099
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 08:00:26.935230 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44467
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:27.038136 30474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:27.038384 30474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:27.038456 30474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:27.042209 30474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:27.042280 30474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:27.042378 30474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:27.047264 30474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:27.048133 30474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:27.049062 30474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:27.057588 30479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:27.057622 30480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:27.057622 30482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:27.057703 30474 server_base.cc:1034] running on GCE node
I20260621 08:00:27.058377 30474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:27.059027 30474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:27.060202 30474 hybrid_clock.cc:648] HybridClock initialized: now 1782028827060194 us; error 38 us; skew 500 ppm
Jun 21 08:00:27 dist-test-slave-0pn3 krb5kdc[30233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028827, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:27.062743 30474 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 08:00:27.063491 30474 webserver.cc:460] Webserver started at http://127.11.223.130:46719/ using document root <none> and password file <none>
I20260621 08:00:27.064013 30474 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:27.064098 30474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:27.064352 30474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:27.066182 30474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "1807a285a6fa4fb097ac4e998116d261"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "47f0f8ba223217b5860cca7db955bdd3"
server_key_iv: "7eda6a5b0dc9e957d23feb6e740c4371"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.066697 30474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "1807a285a6fa4fb097ac4e998116d261"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "47f0f8ba223217b5860cca7db955bdd3"
server_key_iv: "7eda6a5b0dc9e957d23feb6e740c4371"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.070048 30474 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 08:00:27.072197 30489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.073334 30474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:27.073441 30474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "1807a285a6fa4fb097ac4e998116d261"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "47f0f8ba223217b5860cca7db955bdd3"
server_key_iv: "7eda6a5b0dc9e957d23feb6e740c4371"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.073535 30474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:27.084972 30474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:27.087378 30474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:27.087538 30474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:27.087996 30474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:27.088750 30474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:27.088811 30474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.088869 30474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:27.088912 30474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.099633 30474 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:39431
I20260621 08:00:27.099699 30602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:39431 every 8 connection(s)
I20260621 08:00:27.100553 30474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
Jun 21 08:00:27 dist-test-slave-0pn3 krb5kdc[30233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028827, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:27.110500 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30474
I20260621 08:00:27.110633 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 08:00:27.110992 12158 external_mini_cluster.cc:1428] Setting key 6ddad29008183d9fac26e057937f97f9
I20260621 08:00:27.112516 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.102211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:35955 (local address 127.11.223.190:44467)
0621 08:00:27.102354 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.102358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.103074 (+ 716us) server_negotiation.cc:407] Connection header received
0621 08:00:27.104208 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.104210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.104253 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.104377 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:27.105397 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.105890 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.106901 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.107094 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.107170 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.109232 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:27.109235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:27.109237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:27.109258 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:27.110421 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:27.110965 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:27.110968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:27.110969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:27.111056 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:27.111437 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:27.111440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:27.111442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:27.111662 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:27.111760 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.112240 (+ 480us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.112396 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 08:00:27.113464 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.102551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44467 (local address 127.11.223.130:35955)
0621 08:00:27.102944 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.102981 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.104035 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.104525 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.104531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.104845 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:27.105281 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.105293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.105986 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.105989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.106800 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.106807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.107206 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.107208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.107327 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.107781 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:27.107796 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:27.109097 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:27.110531 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:27.110538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:27.110551 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:27.110876 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:27.111179 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:27.111182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:27.111184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:27.111334 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:27.111773 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:27.111780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:27.111990 (+ 210us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.112182 (+ 192us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.112523 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260621 08:00:27.114666 30603 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44467
I20260621 08:00:27.114899 30603 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:27.115447 30603 heartbeater.cc:511] Master 127.11.223.190:44467 requested a full tablet report, sending...
I20260621 08:00:27.116458 30278 ts_manager.cc:194] Registered new tserver with Master: 1807a285a6fa4fb097ac4e998116d261 (127.11.223.130:39431)
I20260621 08:00:27.117013 30278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:35955
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 08:00:27.136056 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44467
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:27.244004 30608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:27.244221 30608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:27.244259 30608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:27.247949 30608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:27.247994 30608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:27.248067 30608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:27.252874 30608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:27.253657 30608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:27.254536 30608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:27.263372 30616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:27.263375 30614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:27.263384 30613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:27.263665 30608 server_base.cc:1034] running on GCE node
I20260621 08:00:27.264107 30608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:27.264724 30608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:27.265909 30608 hybrid_clock.cc:648] HybridClock initialized: now 1782028827265896 us; error 25 us; skew 500 ppm
Jun 21 08:00:27 dist-test-slave-0pn3 krb5kdc[30233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028827, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:27.268576 30608 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 08:00:27.269371 30608 webserver.cc:460] Webserver started at http://127.11.223.131:40209/ using document root <none> and password file <none>
I20260621 08:00:27.269881 30608 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:27.269938 30608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:27.270169 30608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:27.272027 30608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "e3856ab0038a4ddcac026dbf906a8340"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "f99ad4245267a331d3a4b2d6f907e307"
server_key_iv: "be43353358e4424eb46d359e7f4f46b8"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.272603 30608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "e3856ab0038a4ddcac026dbf906a8340"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "f99ad4245267a331d3a4b2d6f907e307"
server_key_iv: "be43353358e4424eb46d359e7f4f46b8"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.275820 30608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:27.278023 30623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.279142 30608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:27.279227 30608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "e3856ab0038a4ddcac026dbf906a8340"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "f99ad4245267a331d3a4b2d6f907e307"
server_key_iv: "be43353358e4424eb46d359e7f4f46b8"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.279340 30608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:27.297041 30608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:27.299573 30608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:27.299752 30608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:27.300303 30608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:27.301064 30608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:27.301115 30608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.301178 30608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:27.301216 30608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.309518 30608 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:35267
I20260621 08:00:27.309541 30736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:35267 every 8 connection(s)
I20260621 08:00:27.310293 30608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:27.311759 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30608
I20260621 08:00:27.311887 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 08:00:27.312197 12158 external_mini_cluster.cc:1428] Setting key d3b0fe0e784d891bf98e98fcd32dc92d
Jun 21 08:00:27 dist-test-slave-0pn3 krb5kdc[30233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028827, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:27.322391 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.312608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:55763 (local address 127.11.223.190:44467)
0621 08:00:27.312744 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.312749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.312910 (+ 161us) server_negotiation.cc:407] Connection header received
0621 08:00:27.314157 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.314160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.314216 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.314291 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:27.315324 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.315870 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.317008 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.317253 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.317361 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.319533 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:27.319536 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:27.319538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:27.319560 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:27.320796 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:27.321241 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:27.321243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:27.321245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:27.321313 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:27.321586 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:27.321587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:27.321589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:27.321734 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:27.321805 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.322202 (+ 397us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.322283 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260621 08:00:27.323087 30739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.312311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:44467 (local address 127.11.223.131:55763)
0621 08:00:27.312769 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.312816 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.313940 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.314441 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.314448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.314759 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:27.315183 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.315195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.315987 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.315990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.316873 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.316882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.317404 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.317408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.317564 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.317997 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:27.318012 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:27.319394 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:27.320897 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:27.320901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:27.320910 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:27.321159 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:27.321386 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:27.321388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:27.321389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:27.321506 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:27.321814 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:27.321818 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:27.322023 (+ 205us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.322159 (+ 136us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.322366 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":121,"threads_started":1}
I20260621 08:00:27.324040 30737 heartbeater.cc:347] Connected to a master server at 127.11.223.190:44467
I20260621 08:00:27.324249 30737 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:27.324690 30737 heartbeater.cc:511] Master 127.11.223.190:44467 requested a full tablet report, sending...
I20260621 08:00:27.325647 30277 ts_manager.cc:194] Registered new tserver with Master: e3856ab0038a4ddcac026dbf906a8340 (127.11.223.131:35267)
I20260621 08:00:27.326189 30277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:55763
I20260621 08:00:27.337255 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 08:00:27.346354 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.339454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.11.223.190:44467)
0621 08:00:27.339569 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.339573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.339730 (+ 157us) server_negotiation.cc:407] Connection header received
0621 08:00:27.339807 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.339809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.339842 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.339907 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:27.340604 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.341077 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.341840 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.342035 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.342109 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.343573 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:27.343576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:27.343578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:27.343598 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:27.344915 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:27.345360 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:27.345363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:27.345364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:27.345431 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:27.345688 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:27.345691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:27.345692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:27.345853 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:27.345958 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.346176 (+ 218us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.346246 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 08:00:27.349216 30277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 08:00:27.352173 30277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 08:00:27.365795 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.359886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:39431 (local address 127.0.0.1:39388)
0621 08:00:27.360372 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.360416 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.360556 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.361130 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.361136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.361168 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:27.361293 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.361300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.362602 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.362607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.363853 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.363863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.365210 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.365214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.365390 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.365426 (+ 36us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.365529 (+ 103us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.365681 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":44,"thread_start_us":75,"threads_started":1}
I20260621 08:00:27.365825 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.360702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.11.223.129:39465)
0621 08:00:27.361137 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.361147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.361167 (+ 20us) server_negotiation.cc:407] Connection header received
0621 08:00:27.361234 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.361238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.361408 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.361565 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:27.361977 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.363100 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.364219 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.365102 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.365244 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.365373 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.365583 (+ 210us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.365679 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":133,"threads_started":1}
I20260621 08:00:27.366209 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.359999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.11.223.130:39431)
0621 08:00:27.360415 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.360427 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.360552 (+ 125us) server_negotiation.cc:407] Connection header received
0621 08:00:27.360729 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.360737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.360934 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.361122 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:27.361410 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.362458 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.364007 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.364898 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.365011 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.365144 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.365983 (+ 839us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.366067 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":102,"threads_started":1}
I20260621 08:00:27.366385 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.359989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39465 (local address 127.0.0.1:59894)
0621 08:00:27.360881 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.360894 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.360973 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.361742 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.361745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.361764 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:27.361858 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.361863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.363234 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.363237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.364059 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.364070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.365243 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.365246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.365419 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.365441 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.365553 (+ 112us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.366104 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":482,"thread_start_us":86,"threads_started":1}
I20260621 08:00:27.367026 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.360649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:35267 (local address 127.0.0.1:33832)
0621 08:00:27.362066 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.362078 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.362169 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.362798 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.362800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.362816 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:27.362907 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.362912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.364402 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.364405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.365568 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.365576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.366645 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.366649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.366742 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.366764 (+ 22us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.366846 (+ 82us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.366907 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":88,"threads_started":1}
I20260621 08:00:27.367029 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.361032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.11.223.131:35267)
0621 08:00:27.361606 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.361617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.362264 (+ 647us) server_negotiation.cc:407] Connection header received
0621 08:00:27.362335 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.362340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.362507 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.362649 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:27.363029 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.364118 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.365682 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.366521 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.366660 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.366735 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.366864 (+ 129us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.366922 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":149,"threads_started":1}
I20260621 08:00:27.369194 30537 tablet_service.cc:1467] Processing CreateTablet for tablet cb73ad3192bd4fb481a42dc31278a633 (DEFAULT_TABLE table=test-table [id=f9a1ce7aea6b44fc83e6ade11dbbdf02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:27.369710 30671 tablet_service.cc:1467] Processing CreateTablet for tablet cb73ad3192bd4fb481a42dc31278a633 (DEFAULT_TABLE table=test-table [id=f9a1ce7aea6b44fc83e6ade11dbbdf02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:27.369707 30403 tablet_service.cc:1467] Processing CreateTablet for tablet cb73ad3192bd4fb481a42dc31278a633 (DEFAULT_TABLE table=test-table [id=f9a1ce7aea6b44fc83e6ade11dbbdf02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:27.370164 30537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb73ad3192bd4fb481a42dc31278a633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:27.370668 30403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb73ad3192bd4fb481a42dc31278a633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:27.370676 30671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb73ad3192bd4fb481a42dc31278a633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:27.377270 30754 tablet_bootstrap.cc:492] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Bootstrap starting.
I20260621 08:00:27.377861 30756 tablet_bootstrap.cc:492] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Bootstrap starting.
I20260621 08:00:27.379236 30754 tablet_bootstrap.cc:654] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:27.379549 30756 tablet_bootstrap.cc:654] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:27.380441 30754 log.cc:824] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:27.380620 30756 log.cc:824] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:27.380666 30755 tablet_bootstrap.cc:492] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: Bootstrap starting.
I20260621 08:00:27.382428 30754 tablet_bootstrap.cc:492] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: No bootstrap required, opened a new log
I20260621 08:00:27.382442 30756 tablet_bootstrap.cc:492] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: No bootstrap required, opened a new log
I20260621 08:00:27.382604 30754 ts_tablet_manager.cc:1397] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 08:00:27.382607 30756 ts_tablet_manager.cc:1397] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 08:00:27.382718 30755 tablet_bootstrap.cc:654] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:27.383879 30755 log.cc:824] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:27.385881 30756 raft_consensus.cc:348] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.386029 30755 tablet_bootstrap.cc:492] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: No bootstrap required, opened a new log
I20260621 08:00:27.386112 30756 raft_consensus.cc:374] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:27.386198 30756 raft_consensus.cc:729] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3856ab0038a4ddcac026dbf906a8340, State: Initialized, Role: FOLLOWER
I20260621 08:00:27.386217 30755 ts_tablet_manager.cc:1397] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 08:00:27.386457 30754 raft_consensus.cc:348] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.386699 30754 raft_consensus.cc:374] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:27.386639 30756 consensus_queue.cc:260] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.386818 30754 raft_consensus.cc:729] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1807a285a6fa4fb097ac4e998116d261, State: Initialized, Role: FOLLOWER
I20260621 08:00:27.387257 30737 heartbeater.cc:503] Master 127.11.223.190:44467 was elected leader, sending a full tablet report...
I20260621 08:00:27.387270 30754 consensus_queue.cc:260] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.387567 30756 ts_tablet_manager.cc:1428] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 08:00:27.388136 30754 ts_tablet_manager.cc:1428] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 08:00:27.388504 30603 heartbeater.cc:503] Master 127.11.223.190:44467 was elected leader, sending a full tablet report...
I20260621 08:00:27.389942 30755 raft_consensus.cc:348] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.390199 30755 raft_consensus.cc:374] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:27.390246 30755 raft_consensus.cc:729] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5613b165c2b34c448782d947e983b60b, State: Initialized, Role: FOLLOWER
I20260621 08:00:27.390671 30755 consensus_queue.cc:260] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.391299 30469 heartbeater.cc:503] Master 127.11.223.190:44467 was elected leader, sending a full tablet report...
I20260621 08:00:27.391618 30755 ts_tablet_manager.cc:1428] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260621 08:00:27.401674 30470 tablet.cc:2369] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 08:00:27.561192 30738 tablet.cc:2369] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:27.598235 30760 raft_consensus.cc:482] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:27.598445 30760 raft_consensus.cc:504] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.599565 30760 leader_election.cc:288] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1807a285a6fa4fb097ac4e998116d261 (127.11.223.130:39431), 5613b165c2b34c448782d947e983b60b (127.11.223.129:39465)
W20260621 08:00:27.601665 30604 tablet.cc:2369] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:27.603997 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.599989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:38797 (local address 127.11.223.129:39465)
0621 08:00:27.600120 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.600123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.600247 (+ 124us) server_negotiation.cc:407] Connection header received
0621 08:00:27.600511 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.600518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.600578 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.600683 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:27.600951 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.601709 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.603096 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.603477 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.603579 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.603618 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.603799 (+ 181us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.603872 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 08:00:27.604012 30763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.599799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39465 (local address 127.11.223.131:38797)
0621 08:00:27.600148 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.600166 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.600286 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.600691 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.600693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.600723 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:27.600839 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.600844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.601830 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.601833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.602983 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.602988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.603577 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.603579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.603649 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.603686 (+ 37us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.603777 (+ 91us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.603843 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":27,"thread_start_us":129,"threads_started":1}
I20260621 08:00:27.604808 30739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.599718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:39431 (local address 127.11.223.131:38997)
0621 08:00:27.599870 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.599886 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:27.599977 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:27.600456 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:27.600461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:27.600483 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:27.600616 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.600625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.601650 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.601653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.602968 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:27.602979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.604344 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.604349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:27.604461 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.604482 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 08:00:27.604585 (+ 103us) client_negotiation.cc:240] Negotiation successful
0621 08:00:27.604662 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":24}
I20260621 08:00:27.604887 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.599813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:38997 (local address 127.11.223.130:39431)
0621 08:00:27.599945 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.599952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.599968 (+ 16us) server_negotiation.cc:407] Connection header received
0621 08:00:27.600115 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.600120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.600183 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.600295 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:27.600755 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.601527 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.603664 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.604051 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.604202 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.604256 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.604679 (+ 423us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.604760 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260621 08:00:27.605484 30423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb73ad3192bd4fb481a42dc31278a633" candidate_uuid: "e3856ab0038a4ddcac026dbf906a8340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5613b165c2b34c448782d947e983b60b" is_pre_election: true
I20260621 08:00:27.605474 30556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb73ad3192bd4fb481a42dc31278a633" candidate_uuid: "e3856ab0038a4ddcac026dbf906a8340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1807a285a6fa4fb097ac4e998116d261" is_pre_election: true
I20260621 08:00:27.605795 30556 raft_consensus.cc:2454] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3856ab0038a4ddcac026dbf906a8340 in term 0.
I20260621 08:00:27.605818 30423 raft_consensus.cc:2454] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3856ab0038a4ddcac026dbf906a8340 in term 0.
I20260621 08:00:27.606297 30625 leader_election.cc:302] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5613b165c2b34c448782d947e983b60b, e3856ab0038a4ddcac026dbf906a8340; no voters:
I20260621 08:00:27.606570 30760 raft_consensus.cc:2781] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 08:00:27.606643 30760 raft_consensus.cc:482] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:27.606673 30760 raft_consensus.cc:3037] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:27.607685 30760 raft_consensus.cc:504] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.608047 30760 leader_election.cc:288] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [CANDIDATE]: Term 1 election: Requested vote from peers 1807a285a6fa4fb097ac4e998116d261 (127.11.223.130:39431), 5613b165c2b34c448782d947e983b60b (127.11.223.129:39465)
I20260621 08:00:27.608364 30556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb73ad3192bd4fb481a42dc31278a633" candidate_uuid: "e3856ab0038a4ddcac026dbf906a8340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1807a285a6fa4fb097ac4e998116d261"
I20260621 08:00:27.608446 30423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb73ad3192bd4fb481a42dc31278a633" candidate_uuid: "e3856ab0038a4ddcac026dbf906a8340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5613b165c2b34c448782d947e983b60b"
I20260621 08:00:27.608492 30556 raft_consensus.cc:3037] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:27.608583 30423 raft_consensus.cc:3037] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:27.609503 30556 raft_consensus.cc:2454] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3856ab0038a4ddcac026dbf906a8340 in term 1.
I20260621 08:00:27.609541 30423 raft_consensus.cc:2454] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3856ab0038a4ddcac026dbf906a8340 in term 1.
I20260621 08:00:27.609826 30626 leader_election.cc:302] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1807a285a6fa4fb097ac4e998116d261, e3856ab0038a4ddcac026dbf906a8340; no voters:
I20260621 08:00:27.610019 30760 raft_consensus.cc:2781] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:27.610261 30760 raft_consensus.cc:686] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 1 LEADER]: Becoming Leader. State: Replica: e3856ab0038a4ddcac026dbf906a8340, State: Running, Role: LEADER
I20260621 08:00:27.610607 30760 consensus_queue.cc:237] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } }
I20260621 08:00:27.613601 30277 catalog_manager.cc:5495] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3856ab0038a4ddcac026dbf906a8340 (127.11.223.131). New cstate: current_term: 1 leader_uuid: "e3856ab0038a4ddcac026dbf906a8340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3856ab0038a4ddcac026dbf906a8340" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 35267 } health_report { overall_health: HEALTHY } } }
I20260621 08:00:27.656306 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.652801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.11.223.131:35267)
0621 08:00:27.652936 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.652940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.652953 (+ 13us) server_negotiation.cc:407] Connection header received
0621 08:00:27.653112 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.653115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.653164 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.653264 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:00:27.653500 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.654004 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.654855 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.655071 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.655197 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.655488 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:00:27.655855 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:00:27.656019 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.656138 (+ 119us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.656175 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 08:00:27.661779 30423 raft_consensus.cc:1261] T cb73ad3192bd4fb481a42dc31278a633 P 5613b165c2b34c448782d947e983b60b [term 1 FOLLOWER]: Refusing update from remote peer e3856ab0038a4ddcac026dbf906a8340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:27.661779 30556 raft_consensus.cc:1261] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 1 FOLLOWER]: Refusing update from remote peer e3856ab0038a4ddcac026dbf906a8340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:27.662423 30760 consensus_queue.cc:1035] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5613b165c2b34c448782d947e983b60b" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:27.662628 30764 consensus_queue.cc:1035] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1807a285a6fa4fb097ac4e998116d261" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 39431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:27.670800 30769 mvcc.cc:205] Tried to move back new op lower bound from 7299190078096171008 to 7299190077895385088. Current Snapshot: MvccSnapshot[applied={T|T < 7299190078096171008}]
I20260621 08:00:27.670850 30768 mvcc.cc:205] Tried to move back new op lower bound from 7299190078096171008 to 7299190077895385088. Current Snapshot: MvccSnapshot[applied={T|T < 7299190078096171008}]
I20260621 08:00:27.677253 30277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:27.677407 30277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:27.679548 30277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60df14d7616347b298e90d7f30b0cccf: Sending DeleteTablet for 3 replicas of tablet cb73ad3192bd4fb481a42dc31278a633
I20260621 08:00:27.680235 30671 tablet_service.cc:1514] Processing DeleteTablet for tablet cb73ad3192bd4fb481a42dc31278a633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:27 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:33832
I20260621 08:00:27.680235 30403 tablet_service.cc:1514] Processing DeleteTablet for tablet cb73ad3192bd4fb481a42dc31278a633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:27 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:59894
I20260621 08:00:27.680374 30537 tablet_service.cc:1514] Processing DeleteTablet for tablet cb73ad3192bd4fb481a42dc31278a633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:27 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:39388
I20260621 08:00:27.680820 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30340
I20260621 08:00:27.683367 30778 tablet_replica.cc:330] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: stopping tablet replica
I20260621 08:00:27.683636 30778 raft_consensus.cc:2229] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 1 LEADER]: Raft consensus shutting down.
I20260621 08:00:27.683923 30778 raft_consensus.cc:2258] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:27.684988 30778 ts_tablet_manager.cc:1905] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:27.686968 30778 ts_tablet_manager.cc:1918] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:27.687093 30778 log.cc:1196] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/cb73ad3192bd4fb481a42dc31278a633
I20260621 08:00:27.687415 30778 ts_tablet_manager.cc:1939] T cb73ad3192bd4fb481a42dc31278a633 P e3856ab0038a4ddcac026dbf906a8340: Deleting consensus metadata
I20260621 08:00:27.688352 30264 catalog_manager.cc:4841] TS e3856ab0038a4ddcac026dbf906a8340 (127.11.223.131:35267): tablet cb73ad3192bd4fb481a42dc31278a633 (table test-table [id=f9a1ce7aea6b44fc83e6ade11dbbdf02]) successfully deleted
I20260621 08:00:27.688482 30780 tablet_replica.cc:330] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: stopping tablet replica
I20260621 08:00:27.688707 30780 raft_consensus.cc:2229] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:27.688959 30780 raft_consensus.cc:2258] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260621 08:00:27.689471 30263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:39465 (error 108)
I20260621 08:00:27.690191 30780 ts_tablet_manager.cc:1905] T cb73ad3192bd4fb481a42dc31278a633 P 1807a285a6fa4fb097ac4e998116d261: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:27.690402 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.690094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39465 (local address 127.0.0.1:59896)
0621 08:00:27.690265 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.690326 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260621 08:00:27.690551 30263 catalog_manager.cc:4584] TS 5613b165c2b34c448782d947e983b60b (127.11.223.129:39465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb73ad3192bd4fb481a42dc31278a633: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39465: connect: Connection refused (error 111)
I20260621 08:00:27.690941 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30474
I20260621 08:00:27.697899 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30608
I20260621 08:00:27.698352 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.698081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:39431 (local address 127.0.0.1:39402)
0621 08:00:27.698232 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:27.698279 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.130:39431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260621 08:00:27.703531 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30246
2026-06-21T08:00:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2389 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-21T08:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:27Z Disabled control of system clock
I20260621 08:00:27.732846 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33369
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:33369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 08:00:27.834774 30785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:27.834998 30785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:27.835031 30785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:27.838943 30785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:27.838985 30785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:27.839000 30785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:27.839015 30785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:27.839030 30785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:27.843443 30785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:33369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:33369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:27.844291 30785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:27.845561 30785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:27.853551 30793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:27.853520 30791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:27.853545 30785 server_base.cc:1034] running on GCE node
W20260621 08:00:27.853549 30790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:27.854233 30785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:27.855195 30785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:27.856379 30785 hybrid_clock.cc:648] HybridClock initialized: now 1782028827856360 us; error 30 us; skew 500 ppm
I20260621 08:00:27.857925 30785 webserver.cc:460] Webserver started at http://127.11.223.190:44785/ using document root <none> and password file <none>
I20260621 08:00:27.858434 30785 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:27.858491 30785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:27.858711 30785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:27.860607 30785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "ac2145e5367a4d02aad21ba0b119ac23"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "c83dc49534d3a7155109d59af8134254"
server_key_iv: "e4d2f83938fa6d760e9459093545a135"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.861153 30785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "ac2145e5367a4d02aad21ba0b119ac23"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "c83dc49534d3a7155109d59af8134254"
server_key_iv: "e4d2f83938fa6d760e9459093545a135"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.864539 30785 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 08:00:27.866683 30799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:27.867797 30785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:27.867928 30785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "ac2145e5367a4d02aad21ba0b119ac23"
format_stamp: "Formatted at 2026-06-21 08:00:27 on dist-test-slave-0pn3"
server_key: "c83dc49534d3a7155109d59af8134254"
server_key_iv: "e4d2f83938fa6d760e9459093545a135"
server_key_version: "encryptionkey@0"
I20260621 08:00:27.868028 30785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:27.886768 30785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:27.887331 30785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:27.887552 30785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:27.893839 30785 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:33369
I20260621 08:00:27.893872 30851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:33369 every 8 connection(s)
I20260621 08:00:27.894660 30785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:27.897524 30852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:27.897710 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30785
I20260621 08:00:27.897830 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:27.898092 12158 external_mini_cluster.cc:1428] Setting key e217eebf1ef98d3f7b23ffb0d239687e
I20260621 08:00:27.904186 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23: Bootstrap starting.
I20260621 08:00:27.906697 30852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:27.907476 30852 log.cc:824] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:27.907910 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:27.899406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.11.223.190:33369)
0621 08:00:27.900032 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:27.900049 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:27.900089 (+ 40us) server_negotiation.cc:407] Connection header received
0621 08:00:27.900841 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:27.900863 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:27.901332 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:27.901664 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:27.902050 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.903463 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.904449 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:27.905102 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:27.905266 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:27.906035 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:27.906040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:27.906054 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:27.906514 (+ 460us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:27.906580 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:27.906606 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:27.906708 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:27.906804 (+ 96us) server_negotiation.cc:299] Negotiation successful
0621 08:00:27.907052 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":102,"threads_started":1}
I20260621 08:00:27.909276 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23: No bootstrap required, opened a new log
I20260621 08:00:27.912844 30852 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2145e5367a4d02aad21ba0b119ac23" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33369 } }
I20260621 08:00:27.913033 30852 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:27.913100 30852 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac2145e5367a4d02aad21ba0b119ac23, State: Initialized, Role: FOLLOWER
I20260621 08:00:27.913468 30852 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2145e5367a4d02aad21ba0b119ac23" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33369 } }
I20260621 08:00:27.913595 30852 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:27.913676 30852 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:27.913779 30852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:27.914566 30852 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2145e5367a4d02aad21ba0b119ac23" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33369 } }
I20260621 08:00:27.914885 30852 leader_election.cc:302] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac2145e5367a4d02aad21ba0b119ac23; no voters:
I20260621 08:00:27.915153 30852 leader_election.cc:288] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:27.915272 30857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:27.915508 30857 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [term 1 LEADER]: Becoming Leader. State: Replica: ac2145e5367a4d02aad21ba0b119ac23, State: Running, Role: LEADER
I20260621 08:00:27.915798 30857 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2145e5367a4d02aad21ba0b119ac23" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33369 } }
I20260621 08:00:27.916296 30852 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:27.917371 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac2145e5367a4d02aad21ba0b119ac23. Latest consensus state: current_term: 1 leader_uuid: "ac2145e5367a4d02aad21ba0b119ac23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2145e5367a4d02aad21ba0b119ac23" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33369 } } }
I20260621 08:00:27.917469 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:27.917330 30858 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac2145e5367a4d02aad21ba0b119ac23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2145e5367a4d02aad21ba0b119ac23" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 33369 } } }
I20260621 08:00:27.917524 30858 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:27.917838 30866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:27.920751 30866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:27.925356 30866 catalog_manager.cc:1347] Generated new cluster ID: d3eb1c282a394fd4baa5a2017d3aad05
I20260621 08:00:27.925416 30866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:27.930811 30866 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:27.931577 30866 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:27.939577 30866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23: Generated new TSK 0
I20260621 08:00:27.940165 30866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:27.944398 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33369
--builtin_ntp_servers=127.11.223.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 08:00:28.049455 30876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:28.049731 30876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:28.049794 30876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:28.053987 30876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:28.054037 30876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:28.054090 30876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:28.058605 30876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:28.059552 30876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:28.060875 30876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:28.069885 30884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:28.069886 30882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:28.070096 30876 server_base.cc:1034] running on GCE node
W20260621 08:00:28.069963 30881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:28.070665 30876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:28.071264 30876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:28.072382 30876 hybrid_clock.cc:648] HybridClock initialized: now 1782028828072361 us; error 32 us; skew 500 ppm
I20260621 08:00:28.074189 30876 webserver.cc:460] Webserver started at http://127.11.223.129:44831/ using document root <none> and password file <none>
I20260621 08:00:28.074735 30876 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:28.074795 30876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:28.075021 30876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:28.077135 30876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "082dd4841e0d44f5bd95339b7c6ea5ed"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "3293dd816186e536545713a543aec8b1"
server_key_iv: "ec4a2b940e5c14ac58df29d1c52a84e1"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.077673 30876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "082dd4841e0d44f5bd95339b7c6ea5ed"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "3293dd816186e536545713a543aec8b1"
server_key_iv: "ec4a2b940e5c14ac58df29d1c52a84e1"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.081212 30876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:28.083575 30890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.084790 30876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:28.084916 30876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "082dd4841e0d44f5bd95339b7c6ea5ed"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "3293dd816186e536545713a543aec8b1"
server_key_iv: "ec4a2b940e5c14ac58df29d1c52a84e1"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.085031 30876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:28.105930 30876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:28.106549 30876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:28.106742 30876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:28.107270 30876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:28.108253 30876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:28.108322 30876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.108426 30876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:28.108481 30876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.117163 30876 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:39535
I20260621 08:00:28.117205 31003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:39535 every 8 connection(s)
I20260621 08:00:28.117974 30876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 08:00:28.120098 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 30876
I20260621 08:00:28.120239 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 08:00:28.120599 12158 external_mini_cluster.cc:1428] Setting key 18b9f7ab4baccf1c7e7d398f6984e29b
I20260621 08:00:28.122466 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33369
--builtin_ntp_servers=127.11.223.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 08:00:28.127393 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.119749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:47603 (local address 127.11.223.190:33369)
0621 08:00:28.119877 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.119880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.121120 (+ 1240us) server_negotiation.cc:407] Connection header received
0621 08:00:28.122745 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.122748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.122812 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.122891 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:28.123707 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.124285 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.125475 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.125754 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.125858 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.126917 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:28.126921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:28.126924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:28.126936 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:28.126954 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:28.126965 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:28.127044 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.127195 (+ 151us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.127258 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260621 08:00:28.128139 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.120204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33369 (local address 127.11.223.129:47603)
0621 08:00:28.120931 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.121000 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.122473 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.123050 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.123056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.123132 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:28.123566 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.123577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.124435 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.124438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.125357 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.125367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.125890 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.125894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.126058 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.126675 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:28.126677 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:28.126714 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:28.126771 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:28.127057 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:28.127067 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:28.127078 (+ 11us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.127193 (+ 115us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.127441 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
I20260621 08:00:28.129392 31004 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33369
I20260621 08:00:28.129623 31004 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:28.130116 31004 heartbeater.cc:511] Master 127.11.223.190:33369 requested a full tablet report, sending...
I20260621 08:00:28.131537 30816 ts_manager.cc:194] Registered new tserver with Master: 082dd4841e0d44f5bd95339b7c6ea5ed (127.11.223.129:39535)
I20260621 08:00:28.132682 30816 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.129:47603
W20260621 08:00:28.229090 31007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:28.229357 31007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:28.229424 31007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:28.233455 31007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:28.233521 31007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:28.233609 31007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:28.238252 31007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:28.239230 31007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:28.240588 31007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:28.249636 31015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:28.249629 31013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:28.249682 31007 server_base.cc:1034] running on GCE node
W20260621 08:00:28.249631 31012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:28.250257 31007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:28.250861 31007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:28.252007 31007 hybrid_clock.cc:648] HybridClock initialized: now 1782028828251978 us; error 40 us; skew 500 ppm
I20260621 08:00:28.253862 31007 webserver.cc:460] Webserver started at http://127.11.223.130:36067/ using document root <none> and password file <none>
I20260621 08:00:28.254406 31007 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:28.254464 31007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:28.254627 31007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:28.256556 31007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "dae6c5cc20eb44588a087eedc772b998"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "45fa86b2fe68ba793293cd58ee34a198"
server_key_iv: "b50f3c02b341df0981c08fee938cbe10"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.257037 31007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "dae6c5cc20eb44588a087eedc772b998"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "45fa86b2fe68ba793293cd58ee34a198"
server_key_iv: "b50f3c02b341df0981c08fee938cbe10"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.260449 31007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:28.262619 31021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.263756 31007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:28.263881 31007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "dae6c5cc20eb44588a087eedc772b998"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "45fa86b2fe68ba793293cd58ee34a198"
server_key_iv: "b50f3c02b341df0981c08fee938cbe10"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.264003 31007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:28.279261 31007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:28.279827 31007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:28.280035 31007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:28.280652 31007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:28.281530 31007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:28.281601 31007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.281697 31007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:28.281746 31007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.290297 31007 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:36593
I20260621 08:00:28.290326 31134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:36593 every 8 connection(s)
I20260621 08:00:28.291086 31007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 08:00:28.298331 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31007
I20260621 08:00:28.298454 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 08:00:28.298739 12158 external_mini_cluster.cc:1428] Setting key 6fd0ac98d442905318b9e772c41e8bb2
I20260621 08:00:28.299619 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.292806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:49389 (local address 127.11.223.190:33369)
0621 08:00:28.292937 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.292940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.293887 (+ 947us) server_negotiation.cc:407] Connection header received
0621 08:00:28.295169 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.295172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.295217 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.295285 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:28.296053 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.296614 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.297642 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.297835 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.297907 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.298962 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:28.298965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:28.298968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:28.298979 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:28.299001 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:28.299017 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:28.299098 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.299422 (+ 324us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.299483 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 08:00:28.300545 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.293164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33369 (local address 127.11.223.130:49389)
0621 08:00:28.293754 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.293792 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.294994 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.295414 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.295420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.295497 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:28.295941 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.295952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.296724 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.296726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.297546 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.297554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.297947 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.297950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.298072 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.298707 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:28.298709 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:28.298747 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:28.298802 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:28.299280 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:28.299287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:28.299291 (+ 4us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.299414 (+ 123us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.299652 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
I20260621 08:00:28.300848 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33369
--builtin_ntp_servers=127.11.223.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 08:00:28.301574 31135 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33369
I20260621 08:00:28.301792 31135 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:28.302364 31135 heartbeater.cc:511] Master 127.11.223.190:33369 requested a full tablet report, sending...
I20260621 08:00:28.303367 30816 ts_manager.cc:194] Registered new tserver with Master: dae6c5cc20eb44588a087eedc772b998 (127.11.223.130:36593)
I20260621 08:00:28.303923 30816 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.130:49389
W20260621 08:00:28.406602 31138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:28.406876 31138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:28.406949 31138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:28.410913 31138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:28.410995 31138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:28.411082 31138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:28.415621 31138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:33369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:28.416715 31138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:28.418170 31138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:28.427230 31146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:28.427211 31144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:28.427258 31138 server_base.cc:1034] running on GCE node
W20260621 08:00:28.427212 31143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:28.427904 31138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:28.428651 31138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:28.429836 31138 hybrid_clock.cc:648] HybridClock initialized: now 1782028828429808 us; error 42 us; skew 500 ppm
I20260621 08:00:28.431711 31138 webserver.cc:460] Webserver started at http://127.11.223.131:44861/ using document root <none> and password file <none>
I20260621 08:00:28.432267 31138 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:28.432379 31138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:28.432564 31138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:28.434409 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "58acdaa59dca4b1bb181dd25f8707003"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "b0fec8134fae5f343f16759c2892134b"
server_key_iv: "966cb50103e50336b059bb61fe3fcf15"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.434878 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "58acdaa59dca4b1bb181dd25f8707003"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "b0fec8134fae5f343f16759c2892134b"
server_key_iv: "966cb50103e50336b059bb61fe3fcf15"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.438220 31138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260621 08:00:28.440418 31152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.441567 31138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 08:00:28.441689 31138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "58acdaa59dca4b1bb181dd25f8707003"
format_stamp: "Formatted at 2026-06-21 08:00:28 on dist-test-slave-0pn3"
server_key: "b0fec8134fae5f343f16759c2892134b"
server_key_iv: "966cb50103e50336b059bb61fe3fcf15"
server_key_version: "encryptionkey@0"
I20260621 08:00:28.441802 31138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:28.458539 31138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:28.459177 31138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:28.459354 31138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:28.459905 31138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:28.460852 31138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:28.460922 31138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.460992 31138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:28.461043 31138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:28.469681 31138 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:46155
I20260621 08:00:28.469702 31265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:46155 every 8 connection(s)
I20260621 08:00:28.470481 31138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:28.477334 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31138
I20260621 08:00:28.477451 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 08:00:28.477741 12158 external_mini_cluster.cc:1428] Setting key 9ad4e2396584751e153c5fb602b83961
I20260621 08:00:28.479282 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.472188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:42833 (local address 127.11.223.190:33369)
0621 08:00:28.472305 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.472337 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.473333 (+ 996us) server_negotiation.cc:407] Connection header received
0621 08:00:28.474595 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.474598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.474675 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.474754 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:28.475545 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.476036 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.477224 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.477499 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.477593 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.478813 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:28.478815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:28.478817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:28.478827 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:28.478843 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:28.478851 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:28.478925 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.479090 (+ 165us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.479148 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260621 08:00:28.480150 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.472622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:33369 (local address 127.11.223.131:42833)
0621 08:00:28.473205 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.473241 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.474428 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.474906 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.474912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.474990 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:28.475432 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.475443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.476155 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.476161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.477115 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.477128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.477660 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.477665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.477846 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.478508 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:28.478511 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:28.478580 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:28.478657 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:28.478939 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:28.478949 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:28.478955 (+ 6us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.479088 (+ 133us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.479350 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":98,"threads_started":1}
I20260621 08:00:28.481201 31266 heartbeater.cc:347] Connected to a master server at 127.11.223.190:33369
I20260621 08:00:28.481411 31266 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:28.481873 31266 heartbeater.cc:511] Master 127.11.223.190:33369 requested a full tablet report, sending...
I20260621 08:00:28.482729 30816 ts_manager.cc:194] Registered new tserver with Master: 58acdaa59dca4b1bb181dd25f8707003 (127.11.223.131:46155)
I20260621 08:00:28.483227 30816 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.131:42833
I20260621 08:00:28.491166 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 08:00:28.496313 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.493028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.11.223.190:33369)
0621 08:00:28.493144 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.493147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.493319 (+ 172us) server_negotiation.cc:407] Connection header received
0621 08:00:28.493392 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.493394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.493424 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.493482 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:28.493715 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.494204 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.494962 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.495177 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.495241 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.495948 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:28.495969 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:28.495971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:28.495981 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:28.495998 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:28.496005 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:28.496096 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.496186 (+ 90us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.496214 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 08:00:28.498759 30816 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 08:00:28.501685 30816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 08:00:28.515476 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.510234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.11.223.131:46155)
0621 08:00:28.511336 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.511347 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.511367 (+ 20us) server_negotiation.cc:407] Connection header received
0621 08:00:28.511434 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.511440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.511618 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.511758 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:28.511941 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.512793 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.514326 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.514846 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.514978 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.515155 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.515237 (+ 82us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.515325 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":981,"thread_start_us":90,"threads_started":1}
I20260621 08:00:28.515471 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.509861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:46155 (local address 127.0.0.1:33420)
0621 08:00:28.510654 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.510666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.510740 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.511738 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.511741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.511757 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:28.511827 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.511833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.512891 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.512911 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.514213 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.514226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.514969 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.514972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.515074 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.515107 (+ 33us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.515214 (+ 107us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.515325 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":163,"thread_start_us":73,"threads_started":1}
I20260621 08:00:28.515760 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.509625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.11.223.129:39535)
0621 08:00:28.510010 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.510021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.510573 (+ 552us) server_negotiation.cc:407] Connection header received
0621 08:00:28.510652 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.510658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.510861 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.511039 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:28.511329 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.512569 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.514126 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.514926 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.515024 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.515196 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.515503 (+ 307us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.515595 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":98,"threads_started":1}
I20260621 08:00:28.516031 31282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.510592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36348 (local address 127.11.223.130:36593)
0621 08:00:28.510981 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.510993 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.511015 (+ 22us) server_negotiation.cc:407] Connection header received
0621 08:00:28.511087 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.511090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.511226 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.511389 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:28.511572 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.512677 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.514322 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.515041 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.515553 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.515697 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.515742 (+ 45us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.515830 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":132,"threads_started":1}
I20260621 08:00:28.516234 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.509497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39535 (local address 127.0.0.1:59510)
0621 08:00:28.510260 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.510305 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.510433 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.511028 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.511033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.511054 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:28.511186 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.511194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.512683 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.512686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.513979 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.513989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.515279 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.515282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.515366 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.515383 (+ 17us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.515958 (+ 575us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.516027 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":136,"thread_start_us":72,"threads_started":1}
I20260621 08:00:28.516868 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.509580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:36593 (local address 127.0.0.1:36348)
0621 08:00:28.510336 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.510350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.510440 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.511358 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.511361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.511378 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:28.511458 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.511463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.512790 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.512793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.514199 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.514208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.515187 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.515191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.515285 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.515306 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.515352 (+ 46us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.515424 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":79,"thread_start_us":109,"threads_started":1}
I20260621 08:00:28.518680 31200 tablet_service.cc:1467] Processing CreateTablet for tablet e6189ae37e834e698801e63db8c51929 (DEFAULT_TABLE table=test-table [id=f2ef3a2a4a0a4a7c9dd36791a626ef9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:28.519052 31069 tablet_service.cc:1467] Processing CreateTablet for tablet e6189ae37e834e698801e63db8c51929 (DEFAULT_TABLE table=test-table [id=f2ef3a2a4a0a4a7c9dd36791a626ef9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:28.519512 31200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6189ae37e834e698801e63db8c51929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:28.519761 31069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6189ae37e834e698801e63db8c51929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:28.522112 30938 tablet_service.cc:1467] Processing CreateTablet for tablet e6189ae37e834e698801e63db8c51929 (DEFAULT_TABLE table=test-table [id=f2ef3a2a4a0a4a7c9dd36791a626ef9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:28.523002 30938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6189ae37e834e698801e63db8c51929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:28.525266 31283 tablet_bootstrap.cc:492] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Bootstrap starting.
I20260621 08:00:28.527238 31283 tablet_bootstrap.cc:654] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:28.527899 31283 log.cc:824] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:28.528604 31284 tablet_bootstrap.cc:492] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Bootstrap starting.
I20260621 08:00:28.529435 31286 tablet_bootstrap.cc:492] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: Bootstrap starting.
I20260621 08:00:28.529803 31283 tablet_bootstrap.cc:492] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: No bootstrap required, opened a new log
I20260621 08:00:28.529983 31283 ts_tablet_manager.cc:1397] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260621 08:00:28.530550 31284 tablet_bootstrap.cc:654] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:28.531306 31284 log.cc:824] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:28.531365 31286 tablet_bootstrap.cc:654] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:28.532011 31286 log.cc:824] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:28.533182 31284 tablet_bootstrap.cc:492] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: No bootstrap required, opened a new log
I20260621 08:00:28.533361 31284 ts_tablet_manager.cc:1397] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 08:00:28.533792 31283 raft_consensus.cc:348] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.533993 31283 raft_consensus.cc:374] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:28.534076 31283 raft_consensus.cc:729] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58acdaa59dca4b1bb181dd25f8707003, State: Initialized, Role: FOLLOWER
I20260621 08:00:28.534308 31286 tablet_bootstrap.cc:492] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: No bootstrap required, opened a new log
I20260621 08:00:28.534471 31286 ts_tablet_manager.cc:1397] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260621 08:00:28.534539 31283 consensus_queue.cc:260] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.535172 31266 heartbeater.cc:503] Master 127.11.223.190:33369 was elected leader, sending a full tablet report...
I20260621 08:00:28.535470 31283 ts_tablet_manager.cc:1428] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260621 08:00:28.537379 31284 raft_consensus.cc:348] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.537601 31284 raft_consensus.cc:374] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:28.537673 31284 raft_consensus.cc:729] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dae6c5cc20eb44588a087eedc772b998, State: Initialized, Role: FOLLOWER
I20260621 08:00:28.538043 31284 consensus_queue.cc:260] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.539076 31135 heartbeater.cc:503] Master 127.11.223.190:33369 was elected leader, sending a full tablet report...
I20260621 08:00:28.539816 31286 raft_consensus.cc:348] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.540045 31286 raft_consensus.cc:374] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:28.540102 31286 raft_consensus.cc:729] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082dd4841e0d44f5bd95339b7c6ea5ed, State: Initialized, Role: FOLLOWER
I20260621 08:00:28.540539 31286 consensus_queue.cc:260] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.540885 31284 ts_tablet_manager.cc:1428] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260621 08:00:28.541409 31286 ts_tablet_manager.cc:1428] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260621 08:00:28.541563 31004 heartbeater.cc:503] Master 127.11.223.190:33369 was elected leader, sending a full tablet report...
W20260621 08:00:28.541885 31136 tablet.cc:2369] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:28.577914 31291 raft_consensus.cc:482] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:28.578154 31291 raft_consensus.cc:504] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.579151 31291 leader_election.cc:288] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dae6c5cc20eb44588a087eedc772b998 (127.11.223.130:36593), 58acdaa59dca4b1bb181dd25f8707003 (127.11.223.131:46155)
I20260621 08:00:28.583629 31282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.579574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:34315 (local address 127.11.223.130:36593)
0621 08:00:28.579709 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.579715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.580138 (+ 423us) server_negotiation.cc:407] Connection header received
0621 08:00:28.580245 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.580248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.580296 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.580399 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:28.580657 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.581289 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.582591 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.583165 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.583262 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.583316 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.583440 (+ 124us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.583504 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 08:00:28.583629 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.579498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:33351 (local address 127.11.223.131:46155)
0621 08:00:28.579623 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.579629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.579646 (+ 17us) server_negotiation.cc:407] Connection header received
0621 08:00:28.579788 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.579793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.579865 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.579954 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:28.580256 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.581043 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.582480 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.583055 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.583200 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.583271 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.583444 (+ 173us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.583505 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 08:00:28.583654 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.579386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:46155 (local address 127.11.223.129:33351)
0621 08:00:28.579529 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.579545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.579663 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.579978 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.579981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.580001 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:28.580127 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.580144 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.581153 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.581156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.582355 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.582366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.583186 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.583189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.583294 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.583322 (+ 28us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.583422 (+ 100us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.583515 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":26}
I20260621 08:00:28.583663 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.579386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:36593 (local address 127.11.223.129:34315)
0621 08:00:28.580038 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.580050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:28.580142 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:28.580419 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:28.580422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:28.580438 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:28.580540 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.580545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.581384 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.581387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.582494 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:28.582500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.583245 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.583248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:28.583322 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.583357 (+ 35us) client_negotiation.cc:769] Sending connection context
0621 08:00:28.583421 (+ 64us) client_negotiation.cc:240] Negotiation successful
0621 08:00:28.583507 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":245,"thread_start_us":167,"threads_started":1}
I20260621 08:00:28.584278 31089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6189ae37e834e698801e63db8c51929" candidate_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dae6c5cc20eb44588a087eedc772b998" is_pre_election: true
I20260621 08:00:28.584278 31220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6189ae37e834e698801e63db8c51929" candidate_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58acdaa59dca4b1bb181dd25f8707003" is_pre_election: true
I20260621 08:00:28.584569 31089 raft_consensus.cc:2454] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082dd4841e0d44f5bd95339b7c6ea5ed in term 0.
I20260621 08:00:28.584569 31220 raft_consensus.cc:2454] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082dd4841e0d44f5bd95339b7c6ea5ed in term 0.
I20260621 08:00:28.585035 30891 leader_election.cc:302] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082dd4841e0d44f5bd95339b7c6ea5ed, dae6c5cc20eb44588a087eedc772b998; no voters:
I20260621 08:00:28.585312 31291 raft_consensus.cc:2781] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 08:00:28.585397 31291 raft_consensus.cc:482] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:28.585443 31291 raft_consensus.cc:3037] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:28.586391 31291 raft_consensus.cc:504] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.586699 31291 leader_election.cc:288] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [CANDIDATE]: Term 1 election: Requested vote from peers dae6c5cc20eb44588a087eedc772b998 (127.11.223.130:36593), 58acdaa59dca4b1bb181dd25f8707003 (127.11.223.131:46155)
I20260621 08:00:28.586954 31089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6189ae37e834e698801e63db8c51929" candidate_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dae6c5cc20eb44588a087eedc772b998"
I20260621 08:00:28.587015 31220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6189ae37e834e698801e63db8c51929" candidate_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58acdaa59dca4b1bb181dd25f8707003"
I20260621 08:00:28.587097 31089 raft_consensus.cc:3037] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:28.587146 31220 raft_consensus.cc:3037] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:28.588368 31220 raft_consensus.cc:2454] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082dd4841e0d44f5bd95339b7c6ea5ed in term 1.
I20260621 08:00:28.588368 31089 raft_consensus.cc:2454] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082dd4841e0d44f5bd95339b7c6ea5ed in term 1.
I20260621 08:00:28.588680 30894 leader_election.cc:302] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082dd4841e0d44f5bd95339b7c6ea5ed, 58acdaa59dca4b1bb181dd25f8707003; no voters:
I20260621 08:00:28.588863 31291 raft_consensus.cc:2781] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:28.589083 31291 raft_consensus.cc:686] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 1 LEADER]: Becoming Leader. State: Replica: 082dd4841e0d44f5bd95339b7c6ea5ed, State: Running, Role: LEADER
I20260621 08:00:28.589457 31291 consensus_queue.cc:237] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } }
I20260621 08:00:28.592554 30815 catalog_manager.cc:5495] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 082dd4841e0d44f5bd95339b7c6ea5ed (127.11.223.129). New cstate: current_term: 1 leader_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082dd4841e0d44f5bd95339b7c6ea5ed" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 } health_report { overall_health: UNKNOWN } } }
W20260621 08:00:28.619064 31005 tablet.cc:2369] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:28.621145 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.617257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.11.223.129:39535)
0621 08:00:28.617417 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:28.617423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:28.617437 (+ 14us) server_negotiation.cc:407] Connection header received
0621 08:00:28.617622 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:28.617625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:28.617677 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:28.617784 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:00:28.618025 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.618554 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.619616 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:28.619818 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:28.619926 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:28.620223 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:00:28.620718 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:00:28.620877 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:28.620965 (+ 88us) server_negotiation.cc:299] Negotiation successful
0621 08:00:28.620999 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260621 08:00:28.626646 31089 raft_consensus.cc:1261] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 1 FOLLOWER]: Refusing update from remote peer 082dd4841e0d44f5bd95339b7c6ea5ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:28.626652 31220 raft_consensus.cc:1261] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 1 FOLLOWER]: Refusing update from remote peer 082dd4841e0d44f5bd95339b7c6ea5ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:28.627332 31293 consensus_queue.cc:1035] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "dae6c5cc20eb44588a087eedc772b998" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:28.627547 31291 consensus_queue.cc:1035] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "58acdaa59dca4b1bb181dd25f8707003" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:28.635500 31298 mvcc.cc:205] Tried to move back new op lower bound from 7299190082048147456 to 7299190081904988160. Current Snapshot: MvccSnapshot[applied={T|T < 7299190082048147456}]
I20260621 08:00:28.635689 31296 mvcc.cc:205] Tried to move back new op lower bound from 7299190082048147456 to 7299190081904988160. Current Snapshot: MvccSnapshot[applied={T|T < 7299190082048147456}]
I20260621 08:00:28.642941 30815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:28.643070 30815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:28.645517 30815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac2145e5367a4d02aad21ba0b119ac23: Sending DeleteTablet for 3 replicas of tablet e6189ae37e834e698801e63db8c51929
I20260621 08:00:28.646243 30938 tablet_service.cc:1514] Processing DeleteTablet for tablet e6189ae37e834e698801e63db8c51929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:28 UTC) from {username='slave'} at 127.0.0.1:59510
I20260621 08:00:28.646239 31069 tablet_service.cc:1514] Processing DeleteTablet for tablet e6189ae37e834e698801e63db8c51929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:28 UTC) from {username='slave'} at 127.0.0.1:36348
I20260621 08:00:28.646663 31306 tablet_replica.cc:330] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed: stopping tablet replica
I20260621 08:00:28.646683 31200 tablet_service.cc:1514] Processing DeleteTablet for tablet e6189ae37e834e698801e63db8c51929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:28 UTC) from {username='slave'} at 127.0.0.1:33420
I20260621 08:00:28.647099 31307 tablet_replica.cc:330] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: stopping tablet replica
I20260621 08:00:28.647120 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30876
I20260621 08:00:28.647140 31306 raft_consensus.cc:2229] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 1 LEADER]: Raft consensus shutting down.
I20260621 08:00:28.647382 31307 raft_consensus.cc:2229] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:28.647538 31306 raft_consensus.cc:2258] T e6189ae37e834e698801e63db8c51929 P 082dd4841e0d44f5bd95339b7c6ea5ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:28.647720 31307 raft_consensus.cc:2258] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:28.649147 31308 tablet_replica.cc:330] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: stopping tablet replica
I20260621 08:00:28.649179 31307 ts_tablet_manager.cc:1905] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:28.649385 31308 raft_consensus.cc:2229] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:28.649648 31308 raft_consensus.cc:2258] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:28.651155 31308 ts_tablet_manager.cc:1905] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:28.651669 31307 ts_tablet_manager.cc:1918] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:28.651775 31307 log.cc:1196] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/e6189ae37e834e698801e63db8c51929
I20260621 08:00:28.652060 31307 ts_tablet_manager.cc:1939] T e6189ae37e834e698801e63db8c51929 P dae6c5cc20eb44588a087eedc772b998: Deleting consensus metadata
I20260621 08:00:28.653087 30800 catalog_manager.cc:4841] TS dae6c5cc20eb44588a087eedc772b998 (127.11.223.130:36593): tablet e6189ae37e834e698801e63db8c51929 (table test-table [id=f2ef3a2a4a0a4a7c9dd36791a626ef9d]) successfully deleted
I20260621 08:00:28.654384 31308 ts_tablet_manager.cc:1918] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:28.654479 31308 log.cc:1196] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/e6189ae37e834e698801e63db8c51929
I20260621 08:00:28.654775 31308 ts_tablet_manager.cc:1939] T e6189ae37e834e698801e63db8c51929 P 58acdaa59dca4b1bb181dd25f8707003: Deleting consensus metadata
W20260621 08:00:28.655465 30801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:39535 (error 108)
I20260621 08:00:28.655881 30803 catalog_manager.cc:4841] TS 58acdaa59dca4b1bb181dd25f8707003 (127.11.223.131:46155): tablet e6189ae37e834e698801e63db8c51929 (table test-table [id=f2ef3a2a4a0a4a7c9dd36791a626ef9d]) successfully deleted
I20260621 08:00:28.656419 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:28.656137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:39535 (local address 127.0.0.1:59522)
0621 08:00:28.656241 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:28.656295 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260621 08:00:28.656446 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31007
W20260621 08:00:28.657083 30801 catalog_manager.cc:4584] TS 082dd4841e0d44f5bd95339b7c6ea5ed (127.11.223.129:39535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6189ae37e834e698801e63db8c51929: Network error: Client connection negotiation failed: client connection to 127.11.223.129:39535: connect: Connection refused (error 111)
I20260621 08:00:28.661770 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31138
I20260621 08:00:28.667025 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 30785
2026-06-21T08:00:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (962 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:28 dist-test-slave-0pn3 krb5kdc[31312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:28 dist-test-slave-0pn3 krb5kdc[31312](info): set up 2 sockets
Jun 21 08:00:28 dist-test-slave-0pn3 krb5kdc[31312](info): commencing operation
krb5kdc: starting...
W20260621 08:00:29.688261 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:29 dist-test-slave-0pn3 krb5kdc[31312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:29.800719 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:38175
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:40201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:38175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:29.905906 31325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:29.906149 31325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:29.906188 31325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:29.910125 31325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:29.910171 31325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:29.910190 31325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:29.910205 31325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:29.910252 31325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:29.915182 31325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:38175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:38175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:29.916178 31325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:29.917173 31325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:29.925320 31330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:29.925396 31325 server_base.cc:1034] running on GCE node
W20260621 08:00:29.925400 31331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:29.925320 31333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:29.926110 31325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:29.927059 31325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:29.928233 31325 hybrid_clock.cc:648] HybridClock initialized: now 1782028829928207 us; error 38 us; skew 500 ppm
Jun 21 08:00:29 dist-test-slave-0pn3 krb5kdc[31312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028829, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:29.931044 31325 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:29.931843 31325 webserver.cc:460] Webserver started at http://127.11.223.190:35133/ using document root <none> and password file <none>
I20260621 08:00:29.932407 31325 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:29.932469 31325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:29.932682 31325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:29.934563 31325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "aa363c2e8531491bae88d56c6706ff92"
format_stamp: "Formatted at 2026-06-21 08:00:29 on dist-test-slave-0pn3"
server_key: "60d27a1d19a3d3a807f530e963f2c852"
server_key_iv: "b663cb5ca6a601a71b2e1a6a49a8ea3f"
server_key_version: "encryptionkey@0"
I20260621 08:00:29.935109 31325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "aa363c2e8531491bae88d56c6706ff92"
format_stamp: "Formatted at 2026-06-21 08:00:29 on dist-test-slave-0pn3"
server_key: "60d27a1d19a3d3a807f530e963f2c852"
server_key_iv: "b663cb5ca6a601a71b2e1a6a49a8ea3f"
server_key_version: "encryptionkey@0"
I20260621 08:00:29.938616 31325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 08:00:29.940958 31340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:29.942107 31325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:29.942242 31325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "aa363c2e8531491bae88d56c6706ff92"
format_stamp: "Formatted at 2026-06-21 08:00:29 on dist-test-slave-0pn3"
server_key: "60d27a1d19a3d3a807f530e963f2c852"
server_key_iv: "b663cb5ca6a601a71b2e1a6a49a8ea3f"
server_key_version: "encryptionkey@0"
I20260621 08:00:29.942349 31325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:29.963703 31325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:29.966930 31325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:29.967133 31325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:29.973878 31325 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:38175
I20260621 08:00:29.973896 31392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:38175 every 8 connection(s)
I20260621 08:00:29.974784 31325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:29.976133 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31325
I20260621 08:00:29.976269 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:29.976595 12158 external_mini_cluster.cc:1428] Setting key 4af850373389f9822ddf1ac349d8e278
I20260621 08:00:29.977900 31393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:29.984246 31393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92: Bootstrap starting.
Jun 21 08:00:29 dist-test-slave-0pn3 krb5kdc[31312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:29.986835 31393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:29.987751 31393 log.cc:824] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:29.989967 31393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92: No bootstrap required, opened a new log
I20260621 08:00:29.990900 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:29.977916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.11.223.190:38175)
0621 08:00:29.978328 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:29.978349 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:29.978383 (+ 34us) server_negotiation.cc:407] Connection header received
0621 08:00:29.978984 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:29.978998 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:29.979311 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:29.979570 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:29.980510 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:29.981483 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:29.982642 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:29.983057 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:29.983171 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:29.985905 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:29.985911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:29.985925 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:29.985954 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:29.987886 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:29.988402 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:29.988406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:29.988410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:29.988544 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:29.988818 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:29.988821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:29.988823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:29.989297 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:29.989475 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:29.989730 (+ 255us) server_negotiation.cc:299] Negotiation successful
0621 08:00:29.990010 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":131,"threads_started":1}
I20260621 08:00:29.993634 31393 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa363c2e8531491bae88d56c6706ff92" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38175 } }
I20260621 08:00:29.993834 31393 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:29.993867 31393 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa363c2e8531491bae88d56c6706ff92, State: Initialized, Role: FOLLOWER
I20260621 08:00:29.994205 31393 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa363c2e8531491bae88d56c6706ff92" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38175 } }
I20260621 08:00:29.994311 31393 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:29.994354 31393 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:29.994421 31393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:29.995193 31393 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa363c2e8531491bae88d56c6706ff92" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38175 } }
I20260621 08:00:29.995466 31393 leader_election.cc:302] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa363c2e8531491bae88d56c6706ff92; no voters:
I20260621 08:00:29.995713 31393 leader_election.cc:288] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:29.995855 31398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:29.996091 31398 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [term 1 LEADER]: Becoming Leader. State: Replica: aa363c2e8531491bae88d56c6706ff92, State: Running, Role: LEADER
I20260621 08:00:29.996454 31398 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa363c2e8531491bae88d56c6706ff92" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38175 } }
I20260621 08:00:29.996889 31393 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:29.998030 31399 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa363c2e8531491bae88d56c6706ff92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa363c2e8531491bae88d56c6706ff92" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38175 } } }
I20260621 08:00:29.998167 31399 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:29.998508 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa363c2e8531491bae88d56c6706ff92. Latest consensus state: current_term: 1 leader_uuid: "aa363c2e8531491bae88d56c6706ff92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa363c2e8531491bae88d56c6706ff92" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 38175 } } }
I20260621 08:00:29.998862 31407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:29.998646 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:30.001587 31407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:30.006378 31407 catalog_manager.cc:1347] Generated new cluster ID: bf3752b67db84c658dbff0f5b3f41b0b
I20260621 08:00:30.006438 31407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:30.016062 31407 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:30.017264 31407 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:30.024035 31407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92: Generated new TSK 0
I20260621 08:00:30.024804 31407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.223.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
Entry for principal kudu/127.11.223.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab.
I20260621 08:00:30.055284 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38175
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:40201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:30.161536 31419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:30.161761 31419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:30.161820 31419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:30.165685 31419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:30.165732 31419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:30.165824 31419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:30.170722 31419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.129.keytab
--principal=kudu/127.11.223.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:30.171607 31419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:30.172536 31419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:30.181917 31424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:30.181934 31425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:30.181999 31427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:30.182091 31419 server_base.cc:1034] running on GCE node
I20260621 08:00:30.182667 31419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:30.183239 31419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:30.184386 31419 hybrid_clock.cc:648] HybridClock initialized: now 1782028830184375 us; error 28 us; skew 500 ppm
Jun 21 08:00:30 dist-test-slave-0pn3 krb5kdc[31312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:30.187094 31419 init.cc:377] Logged in from keytab as kudu/127.11.223.129@KRBTEST.COM (short username kudu)
I20260621 08:00:30.187912 31419 webserver.cc:460] Webserver started at http://127.11.223.129:41773/ using document root <none> and password file <none>
I20260621 08:00:30.188462 31419 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:30.188524 31419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:30.188678 31419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:30.190601 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "ed684b04db114325bbaad960b39f1fd8"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "724b0405fcfc546d30c4bf72f08f6191"
server_key_iv: "33ca387df5023343a38bd862cd64e203"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.191098 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "ed684b04db114325bbaad960b39f1fd8"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "724b0405fcfc546d30c4bf72f08f6191"
server_key_iv: "33ca387df5023343a38bd862cd64e203"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.194473 31419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 08:00:30.196697 31434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.197922 31419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:30.198007 31419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "ed684b04db114325bbaad960b39f1fd8"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "724b0405fcfc546d30c4bf72f08f6191"
server_key_iv: "33ca387df5023343a38bd862cd64e203"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.198091 31419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:30.218153 31419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:30.221025 31419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:30.221184 31419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:30.221655 31419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:30.222364 31419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:30.222409 31419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.222445 31419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:30.222468 31419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.230814 31419 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:35175
I20260621 08:00:30.230857 31547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:35175 every 8 connection(s)
I20260621 08:00:30.231613 31419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
Jun 21 08:00:30 dist-test-slave-0pn3 krb5kdc[31312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.129@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:30.240942 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31419
I20260621 08:00:30.241076 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 08:00:30.241341 12158 external_mini_cluster.cc:1428] Setting key 58612e2fd6d67e471aee9558daa54bbb
I20260621 08:00:30.245009 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.233252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:55501 (local address 127.11.223.190:38175)
0621 08:00:30.233379 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.233389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.234056 (+ 667us) server_negotiation.cc:407] Connection header received
0621 08:00:30.235389 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.235392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.235450 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.235550 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:30.236654 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.237219 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.238348 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.238611 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.238722 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.241226 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:30.241230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:30.241232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:30.241262 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:30.242957 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.243452 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.243457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.243459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.243549 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.243867 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.243871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.243873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.244055 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:30.244179 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.244698 (+ 519us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.244817 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 08:00:30.245803 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.233524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38175 (local address 127.11.223.129:55501)
0621 08:00:30.233921 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.233959 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.235168 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.235720 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.235726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.236030 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:30.236520 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.236533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.237353 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.237356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.238224 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.238232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.238757 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.238761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.238922 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.239573 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:30.239589 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:30.241071 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:30.243097 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:30.243103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:30.243115 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:30.243351 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:30.243644 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:30.243647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:30.243649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:30.243773 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:30.244176 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:30.244183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:30.244468 (+ 285us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.244658 (+ 190us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.244949 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260621 08:00:30.247037 31548 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38175
I20260621 08:00:30.247270 31548 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:30.247704 31548 heartbeater.cc:511] Master 127.11.223.190:38175 requested a full tablet report, sending...
I20260621 08:00:30.249094 31357 ts_manager.cc:194] Registered new tserver with Master: ed684b04db114325bbaad960b39f1fd8 (127.11.223.129:35175)
I20260621 08:00:30.250138 31357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.129@KRBTEST.COM'} at 127.11.223.129:55501
WARNING: no policy specified for kudu/127.11.223.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
Entry for principal kudu/127.11.223.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab.
I20260621 08:00:30.265223 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38175
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:40201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:30.370327 31553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:30.370569 31553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:30.370608 31553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:30.374471 31553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:30.374516 31553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:30.374596 31553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:30.379842 31553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.130.keytab
--principal=kudu/127.11.223.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:30.380803 31553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:30.381747 31553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:30.390800 31561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:30.390760 31559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:30.390827 31553 server_base.cc:1034] running on GCE node
W20260621 08:00:30.390767 31558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:30.391446 31553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:30.392006 31553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:30.393184 31553 hybrid_clock.cc:648] HybridClock initialized: now 1782028830393172 us; error 26 us; skew 500 ppm
Jun 21 08:00:30 dist-test-slave-0pn3 krb5kdc[31312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:30.395641 31553 init.cc:377] Logged in from keytab as kudu/127.11.223.130@KRBTEST.COM (short username kudu)
I20260621 08:00:30.396543 31553 webserver.cc:460] Webserver started at http://127.11.223.130:34467/ using document root <none> and password file <none>
I20260621 08:00:30.397082 31553 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:30.397178 31553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:30.397423 31553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:30.399374 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "4b879248a5f84460929f3aa430543899"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "88a7fdf84f60536e18d7f5c08a9fa306"
server_key_iv: "447643a59dfa29f30a8d0c5a8e1863c2"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.399960 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "4b879248a5f84460929f3aa430543899"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "88a7fdf84f60536e18d7f5c08a9fa306"
server_key_iv: "447643a59dfa29f30a8d0c5a8e1863c2"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.403504 31553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260621 08:00:30.405803 31568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.406915 31553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:30.407047 31553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "4b879248a5f84460929f3aa430543899"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "88a7fdf84f60536e18d7f5c08a9fa306"
server_key_iv: "447643a59dfa29f30a8d0c5a8e1863c2"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.407164 31553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:30.420678 31553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:30.423519 31553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:30.423734 31553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:30.424355 31553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:30.425212 31553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:30.425284 31553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.425355 31553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:30.425400 31553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.434432 31553 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:35251
I20260621 08:00:30.434461 31681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:35251 every 8 connection(s)
I20260621 08:00:30.435269 31553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 08:00:30.440670 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31553
I20260621 08:00:30.440810 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 08:00:30.441109 12158 external_mini_cluster.cc:1428] Setting key a28dd7d2654a794432fddfeaa0b5892c
Jun 21 08:00:30 dist-test-slave-0pn3 krb5kdc[31312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.130@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:30.450172 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.437214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:50233 (local address 127.11.223.190:38175)
0621 08:00:30.437358 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.437362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.438366 (+ 1004us) server_negotiation.cc:407] Connection header received
0621 08:00:30.439701 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.439705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.439763 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.439839 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:30.441118 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.441834 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.443386 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.443601 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.443682 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.446261 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:30.446268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:30.446274 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:30.446307 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:30.448084 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.448612 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.448617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.448619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.448712 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.449051 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.449056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.449059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.449284 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:30.449390 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.449850 (+ 460us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.449975 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260621 08:00:30.450801 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.437699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38175 (local address 127.11.223.130:50233)
0621 08:00:30.438198 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.438258 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.439462 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.440003 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.440010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.440421 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:30.440978 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.440993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.441953 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.441956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.443253 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.443269 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.443745 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.443752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.443978 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.444682 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:30.444697 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:30.446087 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:30.448195 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:30.448200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:30.448210 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:30.448513 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:30.448806 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:30.448809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:30.448810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:30.448935 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:30.449394 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:30.449399 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:30.449608 (+ 209us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.449742 (+ 134us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.449958 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.11.223.131@KRBTEST.COM; defaulting to no policy
I20260621 08:00:30.451963 31682 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38175
I20260621 08:00:30.452185 31682 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:30.452692 31682 heartbeater.cc:511] Master 127.11.223.190:38175 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.131@KRBTEST.COM" created.
I20260621 08:00:30.453711 31357 ts_manager.cc:194] Registered new tserver with Master: 4b879248a5f84460929f3aa430543899 (127.11.223.130:35251)
I20260621 08:00:30.454316 31357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.130@KRBTEST.COM'} at 127.11.223.130:50233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
Entry for principal kudu/127.11.223.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab.
I20260621 08:00:30.467547 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38175
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:40201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:30.573793 31687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:30.574016 31687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:30.574062 31687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:30.577919 31687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:30.577970 31687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:30.578045 31687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:30.582931 31687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.131.keytab
--principal=kudu/127.11.223.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:30.583855 31687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:30.585091 31687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:30.594424 31692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:30.594429 31693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:30.594435 31695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:30.594707 31687 server_base.cc:1034] running on GCE node
I20260621 08:00:30.595089 31687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:30.595677 31687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:30.596802 31687 hybrid_clock.cc:648] HybridClock initialized: now 1782028830596789 us; error 38 us; skew 500 ppm
Jun 21 08:00:30 dist-test-slave-0pn3 krb5kdc[31312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:30.599710 31687 init.cc:377] Logged in from keytab as kudu/127.11.223.131@KRBTEST.COM (short username kudu)
I20260621 08:00:30.600560 31687 webserver.cc:460] Webserver started at http://127.11.223.131:43869/ using document root <none> and password file <none>
I20260621 08:00:30.601125 31687 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:30.601229 31687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:30.601464 31687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:30.603500 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "a06dfc088f104077b29e72b79b4b2ac1"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "eb5b992b0329ca07c7b558fb9990bccc"
server_key_iv: "6acf16f6809683455a707d122f3a9b12"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.604038 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "a06dfc088f104077b29e72b79b4b2ac1"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "eb5b992b0329ca07c7b558fb9990bccc"
server_key_iv: "6acf16f6809683455a707d122f3a9b12"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.607553 31687 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260621 08:00:30.609763 31702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.610948 31687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:30.611115 31687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "a06dfc088f104077b29e72b79b4b2ac1"
format_stamp: "Formatted at 2026-06-21 08:00:30 on dist-test-slave-0pn3"
server_key: "eb5b992b0329ca07c7b558fb9990bccc"
server_key_iv: "6acf16f6809683455a707d122f3a9b12"
server_key_version: "encryptionkey@0"
I20260621 08:00:30.611227 31687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:30.623780 31687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:30.628577 31687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:30.629019 31687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:30.629753 31687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:30.631053 31687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:30.631130 31687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.631206 31687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:30.631246 31687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:30.644272 31687 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:36099
I20260621 08:00:30.644345 31815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:36099 every 8 connection(s)
I20260621 08:00:30.645509 31687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:30.653880 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31687
I20260621 08:00:30.654007 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 08:00:30.654338 12158 external_mini_cluster.cc:1428] Setting key c171b3012903e02ded9f72d1b3ba96e6
Jun 21 08:00:30 dist-test-slave-0pn3 krb5kdc[31312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.131@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:30.662426 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.647597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:42819 (local address 127.11.223.190:38175)
0621 08:00:30.647741 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.647744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.648621 (+ 877us) server_negotiation.cc:407] Connection header received
0621 08:00:30.650055 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.650059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.650113 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.650217 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:30.651605 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.652503 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.655328 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.655564 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.655673 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.658865 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:30.658868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:30.658870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:30.658895 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:30.660252 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.660880 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.660885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.660887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.660997 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.661345 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.661348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.661350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.661534 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:30.661640 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.662137 (+ 497us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.662259 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 08:00:30.663312 31818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.647989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:38175 (local address 127.11.223.131:42819)
0621 08:00:30.648461 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.648514 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.649851 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.650380 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.650388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.650883 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:30.651458 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.651471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.652695 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.652700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.655187 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.655201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.655707 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.655711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.655901 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.656635 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:30.656654 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:30.658701 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:30.660425 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:30.660431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:30.660444 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:30.660796 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:30.661072 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:30.661077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:30.661082 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:30.661243 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:30.661653 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:30.661660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:30.661892 (+ 232us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.662072 (+ 180us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.662373 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":95,"threads_started":1}
I20260621 08:00:30.664482 31816 heartbeater.cc:347] Connected to a master server at 127.11.223.190:38175
I20260621 08:00:30.664710 31816 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:30.665177 31816 heartbeater.cc:511] Master 127.11.223.190:38175 requested a full tablet report, sending...
I20260621 08:00:30.666132 31357 ts_manager.cc:194] Registered new tserver with Master: a06dfc088f104077b29e72b79b4b2ac1 (127.11.223.131:36099)
I20260621 08:00:30.666684 31357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.223.131@KRBTEST.COM'} at 127.11.223.131:42819
I20260621 08:00:30.668670 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 08:00:30.677958 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.670560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.11.223.190:38175)
0621 08:00:30.670695 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.670700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.670892 (+ 192us) server_negotiation.cc:407] Connection header received
0621 08:00:30.670998 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.671000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.671035 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.671128 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:30.671773 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.672264 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.673129 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.673347 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.673454 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.674762 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:30.674766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:30.674768 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:30.674797 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:30.676261 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.676757 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.676762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.676764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.676847 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:30.677113 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:30.677118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:30.677119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:30.677322 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:30.677466 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.677700 (+ 234us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.677803 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 08:00:30.680701 31357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 08:00:30.683485 31357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 08:00:30.697445 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.691365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.11.223.129:35175)
0621 08:00:30.692127 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.692142 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.692168 (+ 26us) server_negotiation.cc:407] Connection header received
0621 08:00:30.692253 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.692258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.692497 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.692970 (+ 473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:30.693149 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.694330 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.695878 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.696708 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.696826 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.696941 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.697132 (+ 191us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.697217 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":147,"threads_started":1}
I20260621 08:00:30.697458 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.691227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:35175 (local address 127.0.0.1:44250)
0621 08:00:30.691900 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.691948 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.692069 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.692666 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.692672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.692698 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:30.692823 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.692829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.694477 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.694481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.695753 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.695763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.696833 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.696839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.696953 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.696986 (+ 33us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.697103 (+ 117us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.697324 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":154,"thread_start_us":103,"threads_started":1}
I20260621 08:00:30.697804 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.691497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:35251 (local address 127.0.0.1:53376)
0621 08:00:30.692334 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.692347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.692421 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.693093 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.693095 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.693111 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:30.693185 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.693189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.694497 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.694500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.695976 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.695985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.697087 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.697093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.697204 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.697251 (+ 47us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.697319 (+ 68us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.697405 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":567,"thread_start_us":94,"threads_started":1}
I20260621 08:00:30.698400 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.691689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.11.223.130:35251)
0621 08:00:30.692679 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.692692 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.692712 (+ 20us) server_negotiation.cc:407] Connection header received
0621 08:00:30.692782 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.692786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.692960 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.693101 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:30.693253 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.694354 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.696117 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.696792 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.696909 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.697026 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.698159 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.698265 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":108,"threads_started":1}
I20260621 08:00:30.700063 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.691948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36099 (local address 127.0.0.1:35378)
0621 08:00:30.694602 (+ 2654us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.694617 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.694712 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.696141 (+ 1429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.696143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.696161 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:30.696254 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.696259 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.697616 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.697619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.698776 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.698785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.699698 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.699701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.699789 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.699810 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.699880 (+ 70us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.699951 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2577,"thread_start_us":94,"threads_started":1}
I20260621 08:00:30.700188 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.692349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.11.223.131:36099)
0621 08:00:30.692688 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.692698 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.694788 (+ 2090us) server_negotiation.cc:407] Connection header received
0621 08:00:30.694865 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.694870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.695002 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.695170 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:30.696399 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.697407 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.698916 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.699582 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.699710 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.699813 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.699953 (+ 140us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.700036 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":151,"threads_started":1}
I20260621 08:00:30.700853 31616 tablet_service.cc:1467] Processing CreateTablet for tablet 43e16e7b9deb475ab343da6c5be5df94 (DEFAULT_TABLE table=test-table [id=01b702dfb12a4b5cbc2879e0116a3f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:30.700853 31482 tablet_service.cc:1467] Processing CreateTablet for tablet 43e16e7b9deb475ab343da6c5be5df94 (DEFAULT_TABLE table=test-table [id=01b702dfb12a4b5cbc2879e0116a3f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:30.701968 31482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e16e7b9deb475ab343da6c5be5df94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:30.701968 31616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e16e7b9deb475ab343da6c5be5df94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:30.702585 31750 tablet_service.cc:1467] Processing CreateTablet for tablet 43e16e7b9deb475ab343da6c5be5df94 (DEFAULT_TABLE table=test-table [id=01b702dfb12a4b5cbc2879e0116a3f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:30.703466 31750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e16e7b9deb475ab343da6c5be5df94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:30.708523 31833 tablet_bootstrap.cc:492] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Bootstrap starting.
I20260621 08:00:30.710007 31835 tablet_bootstrap.cc:492] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: Bootstrap starting.
I20260621 08:00:30.711081 31833 tablet_bootstrap.cc:654] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:30.711663 31835 tablet_bootstrap.cc:654] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:30.712267 31833 log.cc:824] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:30.712671 31834 tablet_bootstrap.cc:492] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Bootstrap starting.
I20260621 08:00:30.712817 31835 log.cc:824] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:30.714268 31833 tablet_bootstrap.cc:492] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: No bootstrap required, opened a new log
I20260621 08:00:30.714442 31833 ts_tablet_manager.cc:1397] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260621 08:00:30.714653 31834 tablet_bootstrap.cc:654] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:30.715142 31835 tablet_bootstrap.cc:492] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: No bootstrap required, opened a new log
I20260621 08:00:30.715312 31835 ts_tablet_manager.cc:1397] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260621 08:00:30.715701 31834 log.cc:824] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:30.717190 31834 tablet_bootstrap.cc:492] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: No bootstrap required, opened a new log
I20260621 08:00:30.717375 31834 ts_tablet_manager.cc:1397] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 08:00:30.718204 31833 raft_consensus.cc:348] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.718465 31833 raft_consensus.cc:374] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:30.718516 31833 raft_consensus.cc:729] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06dfc088f104077b29e72b79b4b2ac1, State: Initialized, Role: FOLLOWER
I20260621 08:00:30.718954 31833 consensus_queue.cc:260] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.718943 31835 raft_consensus.cc:348] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.719172 31835 raft_consensus.cc:374] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:30.719254 31835 raft_consensus.cc:729] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed684b04db114325bbaad960b39f1fd8, State: Initialized, Role: FOLLOWER
I20260621 08:00:30.719631 31835 consensus_queue.cc:260] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.719961 31833 ts_tablet_manager.cc:1428] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260621 08:00:30.720264 31816 heartbeater.cc:503] Master 127.11.223.190:38175 was elected leader, sending a full tablet report...
I20260621 08:00:30.720273 31548 heartbeater.cc:503] Master 127.11.223.190:38175 was elected leader, sending a full tablet report...
I20260621 08:00:30.720520 31835 ts_tablet_manager.cc:1428] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260621 08:00:30.720667 31834 raft_consensus.cc:348] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.720875 31834 raft_consensus.cc:374] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:30.720918 31834 raft_consensus.cc:729] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b879248a5f84460929f3aa430543899, State: Initialized, Role: FOLLOWER
I20260621 08:00:30.721331 31834 consensus_queue.cc:260] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.722151 31834 ts_tablet_manager.cc:1428] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260621 08:00:30.722189 31682 heartbeater.cc:503] Master 127.11.223.190:38175 was elected leader, sending a full tablet report...
W20260621 08:00:30.732494 31549 tablet.cc:2369] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:30.808720 31841 raft_consensus.cc:482] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:30.808949 31841 raft_consensus.cc:504] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.810026 31841 leader_election.cc:288] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed684b04db114325bbaad960b39f1fd8 (127.11.223.129:35175), a06dfc088f104077b29e72b79b4b2ac1 (127.11.223.131:36099)
I20260621 08:00:30.813741 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.810289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:35175 (local address 127.11.223.130:48623)
0621 08:00:30.810418 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.810435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.810531 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.810807 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.810810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.810827 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:30.810955 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.810961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.811837 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.811843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.812710 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.812719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.813329 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.813333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.813433 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.813460 (+ 27us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.813545 (+ 85us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.813620 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260621 08:00:30.813742 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.810377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:48623 (local address 127.11.223.129:35175)
0621 08:00:30.810492 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.810495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.810509 (+ 14us) server_negotiation.cc:407] Connection header received
0621 08:00:30.810653 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.810656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.810711 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.810797 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:30.811076 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.811728 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.812834 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.813211 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.813336 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.813385 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.813562 (+ 177us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.813628 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260621 08:00:30.814196 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.810626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:36099 (local address 127.11.223.130:32811)
0621 08:00:30.811031 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.811042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:30.811180 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:30.811464 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:30.811466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:30.811482 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:30.811631 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.811637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.812402 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.812408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.813133 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:30.813141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.813745 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.813748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:30.813842 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.813876 (+ 34us) client_negotiation.cc:769] Sending connection context
0621 08:00:30.813954 (+ 78us) client_negotiation.cc:240] Negotiation successful
0621 08:00:30.814034 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":149,"threads_started":1}
I20260621 08:00:30.814218 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.810825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:32811 (local address 127.11.223.131:36099)
0621 08:00:30.810947 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.810951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.811142 (+ 191us) server_negotiation.cc:407] Connection header received
0621 08:00:30.811328 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.811331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.811372 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.811454 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:30.811751 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.812263 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.813256 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.813632 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.813769 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.813818 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.813997 (+ 179us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.814081 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260621 08:00:30.814427 31502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e16e7b9deb475ab343da6c5be5df94" candidate_uuid: "4b879248a5f84460929f3aa430543899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed684b04db114325bbaad960b39f1fd8" is_pre_election: true
I20260621 08:00:30.814677 31502 raft_consensus.cc:2454] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b879248a5f84460929f3aa430543899 in term 0.
I20260621 08:00:30.815235 31572 leader_election.cc:302] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b879248a5f84460929f3aa430543899, ed684b04db114325bbaad960b39f1fd8; no voters:
I20260621 08:00:30.815536 31841 raft_consensus.cc:2781] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 08:00:30.815524 31770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e16e7b9deb475ab343da6c5be5df94" candidate_uuid: "4b879248a5f84460929f3aa430543899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06dfc088f104077b29e72b79b4b2ac1" is_pre_election: true
I20260621 08:00:30.815634 31841 raft_consensus.cc:482] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:30.815681 31841 raft_consensus.cc:3037] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:30.815807 31770 raft_consensus.cc:2454] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b879248a5f84460929f3aa430543899 in term 0.
I20260621 08:00:30.816877 31841 raft_consensus.cc:504] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.817179 31841 leader_election.cc:288] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [CANDIDATE]: Term 1 election: Requested vote from peers ed684b04db114325bbaad960b39f1fd8 (127.11.223.129:35175), a06dfc088f104077b29e72b79b4b2ac1 (127.11.223.131:36099)
I20260621 08:00:30.817541 31502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e16e7b9deb475ab343da6c5be5df94" candidate_uuid: "4b879248a5f84460929f3aa430543899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed684b04db114325bbaad960b39f1fd8"
I20260621 08:00:30.817538 31770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e16e7b9deb475ab343da6c5be5df94" candidate_uuid: "4b879248a5f84460929f3aa430543899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06dfc088f104077b29e72b79b4b2ac1"
I20260621 08:00:30.817656 31770 raft_consensus.cc:3037] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:30.817656 31502 raft_consensus.cc:3037] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:30.818879 31502 raft_consensus.cc:2454] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b879248a5f84460929f3aa430543899 in term 1.
I20260621 08:00:30.818879 31770 raft_consensus.cc:2454] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b879248a5f84460929f3aa430543899 in term 1.
I20260621 08:00:30.819156 31572 leader_election.cc:302] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b879248a5f84460929f3aa430543899, a06dfc088f104077b29e72b79b4b2ac1; no voters:
I20260621 08:00:30.819304 31841 raft_consensus.cc:2781] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:30.819506 31841 raft_consensus.cc:686] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 1 LEADER]: Becoming Leader. State: Replica: 4b879248a5f84460929f3aa430543899, State: Running, Role: LEADER
I20260621 08:00:30.819808 31841 consensus_queue.cc:237] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } }
I20260621 08:00:30.823022 31355 catalog_manager.cc:5495] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b879248a5f84460929f3aa430543899 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "4b879248a5f84460929f3aa430543899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b879248a5f84460929f3aa430543899" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20260621 08:00:30.852559 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.849061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.11.223.130:35251)
0621 08:00:30.849189 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:30.849193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:30.849208 (+ 15us) server_negotiation.cc:407] Connection header received
0621 08:00:30.849374 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:30.849376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:30.849428 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:30.849535 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:00:30.849786 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.850265 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.851128 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:30.851354 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:30.851459 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:30.851760 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:00:30.852147 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:00:30.852273 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:30.852349 (+ 76us) server_negotiation.cc:299] Negotiation successful
0621 08:00:30.852403 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 08:00:30.857853 31502 raft_consensus.cc:1261] T 43e16e7b9deb475ab343da6c5be5df94 P ed684b04db114325bbaad960b39f1fd8 [term 1 FOLLOWER]: Refusing update from remote peer 4b879248a5f84460929f3aa430543899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:30.857879 31770 raft_consensus.cc:1261] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 1 FOLLOWER]: Refusing update from remote peer 4b879248a5f84460929f3aa430543899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:30.858541 31841 consensus_queue.cc:1035] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06dfc088f104077b29e72b79b4b2ac1" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 36099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:30.858726 31843 consensus_queue.cc:1035] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed684b04db114325bbaad960b39f1fd8" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 35175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:30.865921 31847 mvcc.cc:205] Tried to move back new op lower bound from 7299190091187159040 to 7299190091040915456. Current Snapshot: MvccSnapshot[applied={T|T < 7299190091187159040}]
I20260621 08:00:30.867409 31848 mvcc.cc:205] Tried to move back new op lower bound from 7299190091187159040 to 7299190091040915456. Current Snapshot: MvccSnapshot[applied={T|T < 7299190091187159040}]
I20260621 08:00:30.872835 31356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:30.872993 31356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:30.875308 31356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa363c2e8531491bae88d56c6706ff92: Sending DeleteTablet for 3 replicas of tablet 43e16e7b9deb475ab343da6c5be5df94
I20260621 08:00:30.875959 31482 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e16e7b9deb475ab343da6c5be5df94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:30 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:44250
I20260621 08:00:30.876012 31750 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e16e7b9deb475ab343da6c5be5df94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:30 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:35378
I20260621 08:00:30.876070 31616 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e16e7b9deb475ab343da6c5be5df94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:30 UTC) from {username='kudu', principal='kudu/127.11.223.190@KRBTEST.COM'} at 127.0.0.1:53376
I20260621 08:00:30.876461 31856 tablet_replica.cc:330] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: stopping tablet replica
I20260621 08:00:30.876739 31856 raft_consensus.cc:2229] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 1 LEADER]: Raft consensus shutting down.
I20260621 08:00:30.877050 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31419
I20260621 08:00:30.877041 31856 raft_consensus.cc:2258] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:30.877782 31857 tablet_replica.cc:330] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: stopping tablet replica
I20260621 08:00:30.878047 31857 raft_consensus.cc:2229] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:30.878340 31857 raft_consensus.cc:2258] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:30.879307 31856 ts_tablet_manager.cc:1905] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:30.879629 31857 ts_tablet_manager.cc:1905] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:30.882316 31856 ts_tablet_manager.cc:1918] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:30.882467 31856 log.cc:1196] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/43e16e7b9deb475ab343da6c5be5df94
I20260621 08:00:30.882772 31857 ts_tablet_manager.cc:1918] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:30.882869 31857 log.cc:1196] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/43e16e7b9deb475ab343da6c5be5df94
I20260621 08:00:30.882966 31856 ts_tablet_manager.cc:1939] T 43e16e7b9deb475ab343da6c5be5df94 P 4b879248a5f84460929f3aa430543899: Deleting consensus metadata
I20260621 08:00:30.883200 31857 ts_tablet_manager.cc:1939] T 43e16e7b9deb475ab343da6c5be5df94 P a06dfc088f104077b29e72b79b4b2ac1: Deleting consensus metadata
I20260621 08:00:30.884093 31341 catalog_manager.cc:4841] TS 4b879248a5f84460929f3aa430543899 (127.11.223.130:35251): tablet 43e16e7b9deb475ab343da6c5be5df94 (table test-table [id=01b702dfb12a4b5cbc2879e0116a3f46]) successfully deleted
I20260621 08:00:30.884241 31344 catalog_manager.cc:4841] TS a06dfc088f104077b29e72b79b4b2ac1 (127.11.223.131:36099): tablet 43e16e7b9deb475ab343da6c5be5df94 (table test-table [id=01b702dfb12a4b5cbc2879e0116a3f46]) successfully deleted
W20260621 08:00:30.885221 31344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:35175 (error 108)
I20260621 08:00:30.885399 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31553
I20260621 08:00:30.889657 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:30.889416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:35175 (local address 127.0.0.1:44252)
0621 08:00:30.889530 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:30.889593 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:35175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260621 08:00:30.889830 31344 catalog_manager.cc:4584] TS ed684b04db114325bbaad960b39f1fd8 (127.11.223.129:35175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43e16e7b9deb475ab343da6c5be5df94: Network error: Client connection negotiation failed: client connection to 127.11.223.129:35175: connect: Connection refused (error 111)
I20260621 08:00:30.891530 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31687
I20260621 08:00:30.896560 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31325
2026-06-21T08:00:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2229 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-21T08:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:30Z Disabled control of system clock
I20260621 08:00:30.925263 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:40903
--webserver_interface=127.11.223.190
--webserver_port=0
--builtin_ntp_servers=127.11.223.148:40711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:40903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 08:00:31.027932 31862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:31.028137 31862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:31.028187 31862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:31.032562 31862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:31.032608 31862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:31.032624 31862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:31.032639 31862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:31.032655 31862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:31.037114 31862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:40903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:40903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:31.037972 31862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:31.039160 31862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:31.046610 31867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:31.046665 31868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:31.046604 31870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.046907 31862 server_base.cc:1034] running on GCE node
I20260621 08:00:31.047381 31862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:31.048262 31862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:31.049464 31862 hybrid_clock.cc:648] HybridClock initialized: now 1782028831049436 us; error 38 us; skew 500 ppm
I20260621 08:00:31.051111 31862 webserver.cc:460] Webserver started at http://127.11.223.190:41003/ using document root <none> and password file <none>
I20260621 08:00:31.051633 31862 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:31.051692 31862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:31.051906 31862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:31.053855 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "29b183012ea042d1b74fe1374d93d55a"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "3a1736361fe1bdecdaff1b49d70a1336"
server_key_iv: "869203eb6ae93700c4fabf95a4ec6617"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.054402 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "29b183012ea042d1b74fe1374d93d55a"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "3a1736361fe1bdecdaff1b49d70a1336"
server_key_iv: "869203eb6ae93700c4fabf95a4ec6617"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.057839 31862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260621 08:00:31.060081 31876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.061210 31862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:31.061332 31862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "29b183012ea042d1b74fe1374d93d55a"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "3a1736361fe1bdecdaff1b49d70a1336"
server_key_iv: "869203eb6ae93700c4fabf95a4ec6617"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.061432 31862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:31.075736 31862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:31.076401 31862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:31.076586 31862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:31.083046 31928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:40903 every 8 connection(s)
I20260621 08:00:31.083047 31862 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:40903
I20260621 08:00:31.084010 31862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:31.086894 31929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:31.090068 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31862
I20260621 08:00:31.090195 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:31.090459 12158 external_mini_cluster.cc:1428] Setting key 103d1c1c35cb97c6f0d53163fd20391c
I20260621 08:00:31.093111 31929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: Bootstrap starting.
I20260621 08:00:31.095984 31929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:31.096911 31929 log.cc:824] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:31.099040 31929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: No bootstrap required, opened a new log
I20260621 08:00:31.101403 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.091954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56104 (local address 127.11.223.190:40903)
0621 08:00:31.092797 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.092820 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.092863 (+ 43us) server_negotiation.cc:407] Connection header received
0621 08:00:31.093836 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.093859 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.094412 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.094780 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:31.095274 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.096807 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.097999 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.098596 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.098772 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.099568 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:31.099573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:31.099596 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:31.100010 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:31.100073 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:31.100096 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:31.100186 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.100274 (+ 88us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.100532 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":130,"threads_started":1}
I20260621 08:00:31.102921 31929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b183012ea042d1b74fe1374d93d55a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40903 } }
I20260621 08:00:31.103125 31929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:31.103159 31929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29b183012ea042d1b74fe1374d93d55a, State: Initialized, Role: FOLLOWER
I20260621 08:00:31.103632 31929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b183012ea042d1b74fe1374d93d55a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40903 } }
I20260621 08:00:31.103753 31929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:31.103839 31929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:31.103940 31929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:31.104827 31929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b183012ea042d1b74fe1374d93d55a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40903 } }
I20260621 08:00:31.105162 31929 leader_election.cc:302] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29b183012ea042d1b74fe1374d93d55a; no voters:
I20260621 08:00:31.105439 31929 leader_election.cc:288] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:31.105576 31934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:31.105801 31934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [term 1 LEADER]: Becoming Leader. State: Replica: 29b183012ea042d1b74fe1374d93d55a, State: Running, Role: LEADER
I20260621 08:00:31.106076 31934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b183012ea042d1b74fe1374d93d55a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40903 } }
I20260621 08:00:31.106511 31929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:31.107638 31935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29b183012ea042d1b74fe1374d93d55a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b183012ea042d1b74fe1374d93d55a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40903 } } }
I20260621 08:00:31.107766 31935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:31.107625 31936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29b183012ea042d1b74fe1374d93d55a. Latest consensus state: current_term: 1 leader_uuid: "29b183012ea042d1b74fe1374d93d55a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b183012ea042d1b74fe1374d93d55a" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 40903 } } }
I20260621 08:00:31.107978 31936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:31.111061 31949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:31.111146 31949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:31.111222 31943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:31.111843 31943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:31.116770 31943 catalog_manager.cc:1347] Generated new cluster ID: 7dc676dbd0244ac9ae7d5b413c7d90ee
I20260621 08:00:31.116827 31943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:31.132392 31943 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:31.133443 31943 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:31.138720 31943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: Generated new TSK 0
I20260621 08:00:31.139317 31943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:31.144933 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.129:0
--local_ip_for_outbound_sockets=127.11.223.129
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:40903
--builtin_ntp_servers=127.11.223.148:40711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260621 08:00:31.250037 31953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:31.250262 31953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:31.250301 31953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:31.254132 31953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:31.254176 31953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:31.254244 31953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.129
I20260621 08:00:31.258766 31953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.223.129
--webserver_port=0
--tserver_master_addrs=127.11.223.190:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.129
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:31.259845 31953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:31.261137 31953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:31.270440 31959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.270536 31953 server_base.cc:1034] running on GCE node
W20260621 08:00:31.270445 31961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:31.270449 31958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.271016 31953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:31.271607 31953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:31.272770 31953 hybrid_clock.cc:648] HybridClock initialized: now 1782028831272756 us; error 24 us; skew 500 ppm
I20260621 08:00:31.274464 31953 webserver.cc:460] Webserver started at http://127.11.223.129:33359/ using document root <none> and password file <none>
I20260621 08:00:31.275007 31953 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:31.275067 31953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:31.275290 31953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:31.277254 31953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data/instance:
uuid: "075987520b044f148696ee41b8a6cbfd"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "1eb416e2c96a56de8df609b84891dbf6"
server_key_iv: "706e648b413738c95356f4caab85cbbb"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.277787 31953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance:
uuid: "075987520b044f148696ee41b8a6cbfd"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "1eb416e2c96a56de8df609b84891dbf6"
server_key_iv: "706e648b413738c95356f4caab85cbbb"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.281095 31953 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:31.283250 31967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.284426 31953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:31.284533 31953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal
uuid: "075987520b044f148696ee41b8a6cbfd"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "1eb416e2c96a56de8df609b84891dbf6"
server_key_iv: "706e648b413738c95356f4caab85cbbb"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.284633 31953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:31.299139 31953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:31.299744 31953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:31.299928 31953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:31.300577 31953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:31.301486 31953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:31.301538 31953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.301612 31953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:31.301645 31953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.310618 31953 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.129:44471
I20260621 08:00:31.310652 32080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.129:44471 every 8 connection(s)
I20260621 08:00:31.311456 31953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/data/info.pb
I20260621 08:00:31.320140 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.313250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.129:43193 (local address 127.11.223.190:40903)
0621 08:00:31.313392 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.313397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.314472 (+ 1075us) server_negotiation.cc:407] Connection header received
0621 08:00:31.315785 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.315787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.315835 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.315909 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:31.316865 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.317387 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.318490 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.318714 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.318792 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.319753 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:31.319756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:31.319758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:31.319768 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:31.319787 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:31.319794 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:31.319851 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.319980 (+ 129us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.320027 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260621 08:00:31.320739 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 31953
I20260621 08:00:31.320868 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-0/wal/instance
I20260621 08:00:31.321071 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.313689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:40903 (local address 127.11.223.129:43193)
0621 08:00:31.314333 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.314371 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.315613 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.316058 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.316064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.316140 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:31.316740 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.316753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.317493 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.317495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.318330 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.318338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.318842 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.318844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.318979 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.319518 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:31.319520 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:31.319565 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:31.319623 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:31.319865 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:31.319871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:31.319875 (+ 4us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.319962 (+ 87us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.320157 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":119,"threads_started":1}
I20260621 08:00:31.321200 12158 external_mini_cluster.cc:1428] Setting key 349e3cc8e3407cf4a7dc239262bbf1dc
I20260621 08:00:31.322244 32081 heartbeater.cc:347] Connected to a master server at 127.11.223.190:40903
I20260621 08:00:31.322475 32081 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:31.323041 32081 heartbeater.cc:511] Master 127.11.223.190:40903 requested a full tablet report, sending...
I20260621 08:00:31.323129 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.130:0
--local_ip_for_outbound_sockets=127.11.223.130
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:40903
--builtin_ntp_servers=127.11.223.148:40711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 08:00:31.324577 31893 ts_manager.cc:194] Registered new tserver with Master: 075987520b044f148696ee41b8a6cbfd (127.11.223.129:44471)
I20260621 08:00:31.326187 31893 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.129:43193
W20260621 08:00:31.428975 32084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:31.429220 32084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:31.429255 32084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:31.433102 32084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:31.433143 32084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:31.433192 32084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.130
I20260621 08:00:31.437687 32084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.223.130
--webserver_port=0
--tserver_master_addrs=127.11.223.190:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.130
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:31.438604 32084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:31.440400 32084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:31.449874 32089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:31.449901 32090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.449985 32084 server_base.cc:1034] running on GCE node
W20260621 08:00:31.449913 32092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.450534 32084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:31.451112 32084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:31.452265 32084 hybrid_clock.cc:648] HybridClock initialized: now 1782028831452234 us; error 43 us; skew 500 ppm
I20260621 08:00:31.453923 32084 webserver.cc:460] Webserver started at http://127.11.223.130:46535/ using document root <none> and password file <none>
I20260621 08:00:31.454427 32084 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:31.454483 32084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:31.454635 32084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:31.456571 32084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data/instance:
uuid: "67411499372c4b528133d770266807b8"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "d2537b9e2c574c04016884d9faddd0a9"
server_key_iv: "5590b11c3e89737925d98c2997a74f11"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.457031 32084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance:
uuid: "67411499372c4b528133d770266807b8"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "d2537b9e2c574c04016884d9faddd0a9"
server_key_iv: "5590b11c3e89737925d98c2997a74f11"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.460572 32084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 08:00:31.462811 32098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.463941 32084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:31.464102 32084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal
uuid: "67411499372c4b528133d770266807b8"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "d2537b9e2c574c04016884d9faddd0a9"
server_key_iv: "5590b11c3e89737925d98c2997a74f11"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.464216 32084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:31.478346 32084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:31.478947 32084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:31.479143 32084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:31.479717 32084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:31.480696 32084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:31.480770 32084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.480839 32084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:31.480883 32084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.489744 32084 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.130:37955
I20260621 08:00:31.489768 32211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.130:37955 every 8 connection(s)
I20260621 08:00:31.490646 32084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/data/info.pb
I20260621 08:00:31.499450 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.492350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:59841 (local address 127.11.223.190:40903)
0621 08:00:31.492485 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.492489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.493470 (+ 981us) server_negotiation.cc:407] Connection header received
0621 08:00:31.494820 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.494823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.494879 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.494991 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:31.495849 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.496420 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.497535 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.497763 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.497875 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.498940 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:31.498944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:31.498946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:31.498965 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:31.498983 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:31.498991 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:31.499064 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.499227 (+ 163us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.499287 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260621 08:00:31.499606 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 32084
I20260621 08:00:31.499737 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal/instance
I20260621 08:00:31.500026 12158 external_mini_cluster.cc:1428] Setting key f87951b4067d662e2b42aef3d0f7fa83
I20260621 08:00:31.500381 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.492715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:40903 (local address 127.11.223.130:59841)
0621 08:00:31.493338 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.493375 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.494618 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.495167 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.495174 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.495252 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:31.495702 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.495714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.496557 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.496560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.497409 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.497418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.497912 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.497916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.498082 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.498698 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:31.498699 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:31.498735 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:31.498792 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:31.499084 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:31.499098 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:31.499103 (+ 5us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.499227 (+ 124us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.499474 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":101,"threads_started":1}
I20260621 08:00:31.501524 32212 heartbeater.cc:347] Connected to a master server at 127.11.223.190:40903
I20260621 08:00:31.501765 32212 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:31.502190 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.223.131:0
--local_ip_for_outbound_sockets=127.11.223.131
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:40903
--builtin_ntp_servers=127.11.223.148:40711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260621 08:00:31.502274 32212 heartbeater.cc:511] Master 127.11.223.190:40903 requested a full tablet report, sending...
I20260621 08:00:31.503326 31893 ts_manager.cc:194] Registered new tserver with Master: 67411499372c4b528133d770266807b8 (127.11.223.130:37955)
I20260621 08:00:31.503991 31893 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.130:59841
W20260621 08:00:31.608794 32215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:31.609061 32215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:31.609126 32215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:31.613044 32215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:31.613113 32215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:31.613193 32215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.223.131
I20260621 08:00:31.617934 32215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:40711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.223.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.223.131
--webserver_port=0
--tserver_master_addrs=127.11.223.190:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.223.131
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:31.618942 32215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:31.620384 32215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:31.629587 32223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.629678 32215 server_base.cc:1034] running on GCE node
W20260621 08:00:31.629617 32220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:31.629657 32221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:31.630208 32215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:31.630839 32215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:31.632031 32215 hybrid_clock.cc:648] HybridClock initialized: now 1782028831632017 us; error 26 us; skew 500 ppm
I20260621 08:00:31.639918 32215 webserver.cc:460] Webserver started at http://127.11.223.131:42413/ using document root <none> and password file <none>
I20260621 08:00:31.640542 32215 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:31.640633 32215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:31.640851 32215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:31.642758 32215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data/instance:
uuid: "bf90e9a5ea934381b8711bae2513f038"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "e238eaa706c7a9ad970c4d57211102fe"
server_key_iv: "664376c4c389dee8d46e7b833f1d0f58"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.643285 32215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance:
uuid: "bf90e9a5ea934381b8711bae2513f038"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "e238eaa706c7a9ad970c4d57211102fe"
server_key_iv: "664376c4c389dee8d46e7b833f1d0f58"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.646878 32215 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.005s
I20260621 08:00:31.649120 32229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.650242 32215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:31.650367 32215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal
uuid: "bf90e9a5ea934381b8711bae2513f038"
format_stamp: "Formatted at 2026-06-21 08:00:31 on dist-test-slave-0pn3"
server_key: "e238eaa706c7a9ad970c4d57211102fe"
server_key_iv: "664376c4c389dee8d46e7b833f1d0f58"
server_key_version: "encryptionkey@0"
I20260621 08:00:31.650467 32215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:31.669013 32215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:31.669586 32215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:31.669759 32215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:31.670275 32215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260621 08:00:31.671108 32215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260621 08:00:31.671176 32215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.671241 32215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260621 08:00:31.671294 32215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:31.679634 32215 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.131:44589
I20260621 08:00:31.679657 32342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.131:44589 every 8 connection(s)
I20260621 08:00:31.680476 32215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/data/info.pb
I20260621 08:00:31.688575 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 32215
I20260621 08:00:31.688704 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal/instance
I20260621 08:00:31.688764 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.682102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.131:44655 (local address 127.11.223.190:40903)
0621 08:00:31.682264 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.682270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.683224 (+ 954us) server_negotiation.cc:407] Connection header received
0621 08:00:31.684502 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.684505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.684550 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.684627 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:31.685430 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.685933 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.686976 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.687186 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.687260 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.688210 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:31.688213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:31.688215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:31.688226 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:31.688244 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:31.688252 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:31.688352 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.688509 (+ 157us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.688603 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260621 08:00:31.689024 12158 external_mini_cluster.cc:1428] Setting key c812c08d2ced8387bd26677d0b3b28d4
I20260621 08:00:31.689615 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.682494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:40903 (local address 127.11.223.131:44655)
0621 08:00:31.683093 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.683129 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.684331 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.684776 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.684782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.684879 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:31.685319 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.685331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.686033 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.686036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.686878 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.686886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.687301 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.687304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.687437 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.687988 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0621 08:00:31.687989 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:31.688024 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0621 08:00:31.688080 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:31.688378 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:31.688384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:31.688388 (+ 4us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.688478 (+ 90us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.688858 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
I20260621 08:00:31.690721 32343 heartbeater.cc:347] Connected to a master server at 127.11.223.190:40903
I20260621 08:00:31.690950 32343 heartbeater.cc:464] Registering TS with master...
I20260621 08:00:31.691470 32343 heartbeater.cc:511] Master 127.11.223.190:40903 requested a full tablet report, sending...
I20260621 08:00:31.692404 31893 ts_manager.cc:194] Registered new tserver with Master: bf90e9a5ea934381b8711bae2513f038 (127.11.223.131:44589)
I20260621 08:00:31.692962 31893 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.223.131:44655
I20260621 08:00:31.702651 12158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260621 08:00:31.708076 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.704575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.11.223.190:40903)
0621 08:00:31.704682 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.704686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.704830 (+ 144us) server_negotiation.cc:407] Connection header received
0621 08:00:31.704919 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.704921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.704966 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.705031 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:31.705257 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.705737 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.706618 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.706818 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.706891 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.707632 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:31.707635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:31.707638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:31.707656 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:31.707672 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:31.707684 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:31.707761 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.707879 (+ 118us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.707929 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 08:00:31.710748 31893 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260621 08:00:31.713721 31893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260621 08:00:31.727260 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.721292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:44471 (local address 127.0.0.1:53856)
0621 08:00:31.722320 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.722364 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.722528 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.723145 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.723150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.723170 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:31.723305 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.723311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.724875 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.724878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.726013 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.726022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.726761 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.726764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.726862 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.726893 (+ 31us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.726997 (+ 104us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.727103 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260621 08:00:31.727394 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.722070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.130:37955 (local address 127.0.0.1:41062)
0621 08:00:31.722885 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.722895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.723003 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.723436 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.723439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.723454 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:31.723541 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.723547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.724494 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.724500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.725744 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.725753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.727000 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.727004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.727132 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.727152 (+ 20us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.727217 (+ 65us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.727272 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":153,"thread_start_us":84,"threads_started":1}
I20260621 08:00:31.727572 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.721397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:44589 (local address 127.0.0.1:42906)
0621 08:00:31.722300 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.722343 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.722459 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.723168 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.723171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.723195 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:31.723312 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.723317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.724739 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.724742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.725947 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.725957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.727091 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.727093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.727248 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.727269 (+ 21us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.727343 (+ 74us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.727414 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":85,"thread_start_us":109,"threads_started":1}
I20260621 08:00:31.727598 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.721396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.11.223.129:44471)
0621 08:00:31.722079 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.722093 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.722454 (+ 361us) server_negotiation.cc:407] Connection header received
0621 08:00:31.722687 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.722693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.722885 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.723031 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:31.723430 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.724765 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.726119 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.726636 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.726762 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.726928 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.727188 (+ 260us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.727288 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":130,"threads_started":1}
I20260621 08:00:31.727815 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.722297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.11.223.130:37955)
0621 08:00:31.722784 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.722792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.722993 (+ 201us) server_negotiation.cc:407] Connection header received
0621 08:00:31.723127 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.723132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.723316 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.723456 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:31.723637 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.724390 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.725870 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.726665 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.726781 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.726943 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.727574 (+ 631us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.727661 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":143,"threads_started":1}
I20260621 08:00:31.728780 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.721576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.11.223.131:44589)
0621 08:00:31.721984 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.721997 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.722629 (+ 632us) server_negotiation.cc:407] Connection header received
0621 08:00:31.722698 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.722703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.722885 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.723031 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:31.723469 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.724615 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.726089 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.726795 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.726896 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.727022 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.728512 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.728617 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":138,"threads_started":1}
I20260621 08:00:31.730278 32146 tablet_service.cc:1467] Processing CreateTablet for tablet 55aeaa8f8a0f44f1803194037bfdbbf1 (DEFAULT_TABLE table=test-table [id=424ca11449bc4e26ac0ede420bb3d89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:31.730280 32015 tablet_service.cc:1467] Processing CreateTablet for tablet 55aeaa8f8a0f44f1803194037bfdbbf1 (DEFAULT_TABLE table=test-table [id=424ca11449bc4e26ac0ede420bb3d89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:31.731153 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55aeaa8f8a0f44f1803194037bfdbbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:31.731189 32015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55aeaa8f8a0f44f1803194037bfdbbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:31.732970 32277 tablet_service.cc:1467] Processing CreateTablet for tablet 55aeaa8f8a0f44f1803194037bfdbbf1 (DEFAULT_TABLE table=test-table [id=424ca11449bc4e26ac0ede420bb3d89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260621 08:00:31.733867 32277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55aeaa8f8a0f44f1803194037bfdbbf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:31.737041 32360 tablet_bootstrap.cc:492] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Bootstrap starting.
I20260621 08:00:31.738366 32361 tablet_bootstrap.cc:492] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: Bootstrap starting.
I20260621 08:00:31.739173 32360 tablet_bootstrap.cc:654] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:31.739771 32362 tablet_bootstrap.cc:492] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Bootstrap starting.
I20260621 08:00:31.739971 32360 log.cc:824] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:31.740294 32361 tablet_bootstrap.cc:654] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:31.741007 32361 log.cc:824] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:31.741855 32362 tablet_bootstrap.cc:654] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:31.742594 32362 log.cc:824] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:31.742906 32360 tablet_bootstrap.cc:492] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: No bootstrap required, opened a new log
I20260621 08:00:31.743114 32360 ts_tablet_manager.cc:1397] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260621 08:00:31.743571 32361 tablet_bootstrap.cc:492] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: No bootstrap required, opened a new log
I20260621 08:00:31.743743 32361 ts_tablet_manager.cc:1397] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260621 08:00:31.744257 32362 tablet_bootstrap.cc:492] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: No bootstrap required, opened a new log
I20260621 08:00:31.744452 32362 ts_tablet_manager.cc:1397] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260621 08:00:31.747033 32360 raft_consensus.cc:348] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.747315 32360 raft_consensus.cc:374] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:31.747200 32362 raft_consensus.cc:348] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.747375 32360 raft_consensus.cc:729] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67411499372c4b528133d770266807b8, State: Initialized, Role: FOLLOWER
I20260621 08:00:31.747423 32362 raft_consensus.cc:374] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:31.747354 32361 raft_consensus.cc:348] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.747494 32362 raft_consensus.cc:729] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf90e9a5ea934381b8711bae2513f038, State: Initialized, Role: FOLLOWER
I20260621 08:00:31.747567 32361 raft_consensus.cc:374] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:31.747627 32361 raft_consensus.cc:729] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075987520b044f148696ee41b8a6cbfd, State: Initialized, Role: FOLLOWER
I20260621 08:00:31.747840 32360 consensus_queue.cc:260] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.747964 32361 consensus_queue.cc:260] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.747975 32362 consensus_queue.cc:260] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.748637 32081 heartbeater.cc:503] Master 127.11.223.190:40903 was elected leader, sending a full tablet report...
I20260621 08:00:31.748816 32360 ts_tablet_manager.cc:1428] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260621 08:00:31.749116 32212 heartbeater.cc:503] Master 127.11.223.190:40903 was elected leader, sending a full tablet report...
I20260621 08:00:31.749728 32361 ts_tablet_manager.cc:1428] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260621 08:00:31.749818 32362 ts_tablet_manager.cc:1428] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260621 08:00:31.750085 32343 heartbeater.cc:503] Master 127.11.223.190:40903 was elected leader, sending a full tablet report...
W20260621 08:00:31.812659 32082 tablet.cc:2369] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260621 08:00:31.931465 32344 tablet.cc:2369] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:31.951485 32366 raft_consensus.cc:482] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260621 08:00:31.951694 32366 raft_consensus.cc:504] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.952852 32366 leader_election.cc:288] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 075987520b044f148696ee41b8a6cbfd (127.11.223.129:44471), bf90e9a5ea934381b8711bae2513f038 (127.11.223.131:44589)
I20260621 08:00:31.956475 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.953038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:44471 (local address 127.11.223.130:50245)
0621 08:00:31.953226 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.953242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.953330 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.953610 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.953612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.953628 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:31.953772 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.953781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.954612 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.954617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.955375 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.955383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.955985 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.955988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.956102 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.956157 (+ 55us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.956239 (+ 82us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.956308 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":33}
I20260621 08:00:31.956475 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.953160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:50245 (local address 127.11.223.129:44471)
0621 08:00:31.953314 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.953318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.953333 (+ 15us) server_negotiation.cc:407] Connection header received
0621 08:00:31.953468 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.953470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.953512 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.953598 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:31.953891 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.954497 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.955504 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.955870 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.955995 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.956055 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.956265 (+ 210us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.956348 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 08:00:31.956568 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.953177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.223.130:37315 (local address 127.11.223.131:44589)
0621 08:00:31.953412 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:31.953418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:31.953509 (+ 91us) server_negotiation.cc:407] Connection header received
0621 08:00:31.953669 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:31.953672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:31.953710 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:31.953783 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0621 08:00:31.954032 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.954590 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.955562 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.955931 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.956065 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.956121 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:31.956345 (+ 224us) server_negotiation.cc:299] Negotiation successful
0621 08:00:31.956447 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260621 08:00:31.956652 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:31.953084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.131:44589 (local address 127.11.223.130:37315)
0621 08:00:31.953411 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:31.953426 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:31.953561 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:31.953801 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:31.953803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:31.953818 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0621 08:00:31.953931 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.953936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.954691 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.954697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.955445 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:31.955453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:31.956056 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:31.956060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:31.956213 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:31.956228 (+ 15us) client_negotiation.cc:769] Sending connection context
0621 08:00:31.956311 (+ 83us) client_negotiation.cc:240] Negotiation successful
0621 08:00:31.956502 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":40,"thread_start_us":116,"threads_started":1}
I20260621 08:00:31.957237 32297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55aeaa8f8a0f44f1803194037bfdbbf1" candidate_uuid: "67411499372c4b528133d770266807b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf90e9a5ea934381b8711bae2513f038" is_pre_election: true
I20260621 08:00:31.957293 32035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55aeaa8f8a0f44f1803194037bfdbbf1" candidate_uuid: "67411499372c4b528133d770266807b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075987520b044f148696ee41b8a6cbfd" is_pre_election: true
I20260621 08:00:31.957540 32297 raft_consensus.cc:2454] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67411499372c4b528133d770266807b8 in term 0.
I20260621 08:00:31.957585 32035 raft_consensus.cc:2454] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67411499372c4b528133d770266807b8 in term 0.
I20260621 08:00:31.958004 32101 leader_election.cc:302] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67411499372c4b528133d770266807b8, bf90e9a5ea934381b8711bae2513f038; no voters:
I20260621 08:00:31.958245 32366 raft_consensus.cc:2781] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260621 08:00:31.958321 32366 raft_consensus.cc:482] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260621 08:00:31.958362 32366 raft_consensus.cc:3037] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:31.959298 32366 raft_consensus.cc:504] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.959586 32366 leader_election.cc:288] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [CANDIDATE]: Term 1 election: Requested vote from peers 075987520b044f148696ee41b8a6cbfd (127.11.223.129:44471), bf90e9a5ea934381b8711bae2513f038 (127.11.223.131:44589)
I20260621 08:00:31.959820 32035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55aeaa8f8a0f44f1803194037bfdbbf1" candidate_uuid: "67411499372c4b528133d770266807b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075987520b044f148696ee41b8a6cbfd"
I20260621 08:00:31.959923 32297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55aeaa8f8a0f44f1803194037bfdbbf1" candidate_uuid: "67411499372c4b528133d770266807b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf90e9a5ea934381b8711bae2513f038"
I20260621 08:00:31.959924 32035 raft_consensus.cc:3037] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:31.960044 32297 raft_consensus.cc:3037] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:31.961205 32035 raft_consensus.cc:2454] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67411499372c4b528133d770266807b8 in term 1.
I20260621 08:00:31.961248 32297 raft_consensus.cc:2454] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67411499372c4b528133d770266807b8 in term 1.
I20260621 08:00:31.961511 32100 leader_election.cc:302] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075987520b044f148696ee41b8a6cbfd, 67411499372c4b528133d770266807b8; no voters:
I20260621 08:00:31.961678 32366 raft_consensus.cc:2781] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:31.961908 32366 raft_consensus.cc:686] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 1 LEADER]: Becoming Leader. State: Replica: 67411499372c4b528133d770266807b8, State: Running, Role: LEADER
I20260621 08:00:31.962242 32366 consensus_queue.cc:237] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } }
I20260621 08:00:31.965222 31893 catalog_manager.cc:5495] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 67411499372c4b528133d770266807b8 (127.11.223.130). New cstate: current_term: 1 leader_uuid: "67411499372c4b528133d770266807b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67411499372c4b528133d770266807b8" member_type: VOTER last_known_addr { host: "127.11.223.130" port: 37955 } health_report { overall_health: HEALTHY } } }
W20260621 08:00:31.991653 32213 tablet.cc:2369] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:00:32.014881 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:32.011356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.11.223.130:37955)
0621 08:00:32.011490 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:32.011495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:32.011509 (+ 14us) server_negotiation.cc:407] Connection header received
0621 08:00:32.011664 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:32.011667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:32.011732 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:32.011839 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:00:32.012084 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:32.012598 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:32.013439 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:32.013715 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:32.013826 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:32.014129 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:00:32.014515 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:00:32.014644 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:32.014701 (+ 57us) server_negotiation.cc:299] Negotiation successful
0621 08:00:32.014751 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260621 08:00:32.020509 32035 raft_consensus.cc:1261] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 075987520b044f148696ee41b8a6cbfd [term 1 FOLLOWER]: Refusing update from remote peer 67411499372c4b528133d770266807b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:32.020539 32297 raft_consensus.cc:1261] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 1 FOLLOWER]: Refusing update from remote peer 67411499372c4b528133d770266807b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:32.021154 32366 consensus_queue.cc:1035] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "075987520b044f148696ee41b8a6cbfd" member_type: VOTER last_known_addr { host: "127.11.223.129" port: 44471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:32.021356 32370 consensus_queue.cc:1035] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf90e9a5ea934381b8711bae2513f038" member_type: VOTER last_known_addr { host: "127.11.223.131" port: 44589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:32.028445 32373 mvcc.cc:205] Tried to move back new op lower bound from 7299190095948677120 to 7299190095719800832. Current Snapshot: MvccSnapshot[applied={T|T < 7299190095948677120}]
I20260621 08:00:32.029624 32374 mvcc.cc:205] Tried to move back new op lower bound from 7299190095948677120 to 7299190095719800832. Current Snapshot: MvccSnapshot[applied={T|T < 7299190095948677120}]
I20260621 08:00:32.036432 31893 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:32.036562 31893 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260621 08:00:32.038805 31893 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29b183012ea042d1b74fe1374d93d55a: Sending DeleteTablet for 3 replicas of tablet 55aeaa8f8a0f44f1803194037bfdbbf1
I20260621 08:00:32.039536 32277 tablet_service.cc:1514] Processing DeleteTablet for tablet 55aeaa8f8a0f44f1803194037bfdbbf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:32 UTC) from {username='slave'} at 127.0.0.1:42906
I20260621 08:00:32.039849 32146 tablet_service.cc:1514] Processing DeleteTablet for tablet 55aeaa8f8a0f44f1803194037bfdbbf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:32 UTC) from {username='slave'} at 127.0.0.1:41062
I20260621 08:00:32.039980 32015 tablet_service.cc:1514] Processing DeleteTablet for tablet 55aeaa8f8a0f44f1803194037bfdbbf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-21 08:00:32 UTC) from {username='slave'} at 127.0.0.1:53856
I20260621 08:00:32.040268 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31953
I20260621 08:00:32.040360 32384 tablet_replica.cc:330] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: stopping tablet replica
I20260621 08:00:32.041157 32384 raft_consensus.cc:2229] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 1 LEADER]: Raft consensus shutting down.
I20260621 08:00:32.041463 32384 raft_consensus.cc:2258] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:32.042048 32383 tablet_replica.cc:330] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: stopping tablet replica
I20260621 08:00:32.042318 32383 raft_consensus.cc:2229] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260621 08:00:32.042605 32383 raft_consensus.cc:2258] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260621 08:00:32.043454 32384 ts_tablet_manager.cc:1905] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:32.043823 32383 ts_tablet_manager.cc:1905] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260621 08:00:32.046507 32384 ts_tablet_manager.cc:1918] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:32.046666 32384 log.cc:1196] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-1/wal/wals/55aeaa8f8a0f44f1803194037bfdbbf1
I20260621 08:00:32.047070 32384 ts_tablet_manager.cc:1939] T 55aeaa8f8a0f44f1803194037bfdbbf1 P 67411499372c4b528133d770266807b8: Deleting consensus metadata
I20260621 08:00:32.047118 32383 ts_tablet_manager.cc:1918] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260621 08:00:32.047209 32383 log.cc:1196] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Deleting WAL directory at /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1782028587408853-12158-0/minicluster-data/ts-2/wal/wals/55aeaa8f8a0f44f1803194037bfdbbf1
I20260621 08:00:32.047518 32383 ts_tablet_manager.cc:1939] T 55aeaa8f8a0f44f1803194037bfdbbf1 P bf90e9a5ea934381b8711bae2513f038: Deleting consensus metadata
W20260621 08:00:32.047652 31878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.223.129:44471 (error 108)
I20260621 08:00:32.048395 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 32084
I20260621 08:00:32.048506 31880 catalog_manager.cc:4841] TS 67411499372c4b528133d770266807b8 (127.11.223.130:37955): tablet 55aeaa8f8a0f44f1803194037bfdbbf1 (table test-table [id=424ca11449bc4e26ac0ede420bb3d89d]) successfully deleted
I20260621 08:00:32.048884 31879 catalog_manager.cc:4841] TS bf90e9a5ea934381b8711bae2513f038 (127.11.223.131:44589): tablet 55aeaa8f8a0f44f1803194037bfdbbf1 (table test-table [id=424ca11449bc4e26ac0ede420bb3d89d]) successfully deleted
I20260621 08:00:32.049894 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:32.049582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.129:44471 (local address 127.0.0.1:53860)
0621 08:00:32.049725 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:32.049801 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.129:44471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 08:00:32.050067 31878 catalog_manager.cc:4584] TS 075987520b044f148696ee41b8a6cbfd (127.11.223.129:44471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55aeaa8f8a0f44f1803194037bfdbbf1: Network error: Client connection negotiation failed: client connection to 127.11.223.129:44471: connect: Connection refused (error 111)
I20260621 08:00:32.054219 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 32215
I20260621 08:00:32.059453 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 31862
2026-06-21T08:00:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1162 ms)
[----------] 6 tests from EncryptionPolicyTest (10177 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:32 dist-test-slave-0pn3 krb5kdc[32389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:32 dist-test-slave-0pn3 krb5kdc[32389](info): set up 2 sockets
Jun 21 08:00:32 dist-test-slave-0pn3 krb5kdc[32389](info): commencing operation
krb5kdc: starting...
W20260621 08:00:33.072522 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:33 dist-test-slave-0pn3 krb5kdc[32389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:33.190871 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:33.297114 32402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:33.297336 32402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:33.297410 32402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:33.301357 32402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:33.301404 32402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:33.301422 32402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:33.301438 32402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:33.301453 32402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:33.306478 32402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34923
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:33.307471 32402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:33.308477 32402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:33.316270 32410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:33.316340 32408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:33.316363 32407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:33.316617 32402 server_base.cc:1034] running on GCE node
I20260621 08:00:33.317067 32402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:33.318035 32402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:33.319190 32402 hybrid_clock.cc:648] HybridClock initialized: now 1782028833319172 us; error 28 us; skew 500 ppm
Jun 21 08:00:33 dist-test-slave-0pn3 krb5kdc[32389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:33.321672 32402 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:33.322480 32402 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
I20260621 08:00:33.322984 32402 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:33.323073 32402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:33.323294 32402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:33.325241 32402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "4ecf4a42f7bc41199c73f537c6ede43b"
format_stamp: "Formatted at 2026-06-21 08:00:33 on dist-test-slave-0pn3"
server_key: "0024ae27f1342439e9c86423a341b651"
server_key_iv: "ad6db7f96987a600fccaa13717b4dc14"
server_key_version: "encryptionkey@0"
I20260621 08:00:33.325821 32402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "4ecf4a42f7bc41199c73f537c6ede43b"
format_stamp: "Formatted at 2026-06-21 08:00:33 on dist-test-slave-0pn3"
server_key: "0024ae27f1342439e9c86423a341b651"
server_key_iv: "ad6db7f96987a600fccaa13717b4dc14"
server_key_version: "encryptionkey@0"
I20260621 08:00:33.329195 32402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:33.331497 32417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:33.332674 32402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:33.332803 32402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "4ecf4a42f7bc41199c73f537c6ede43b"
format_stamp: "Formatted at 2026-06-21 08:00:33 on dist-test-slave-0pn3"
server_key: "0024ae27f1342439e9c86423a341b651"
server_key_iv: "ad6db7f96987a600fccaa13717b4dc14"
server_key_version: "encryptionkey@0"
I20260621 08:00:33.332906 32402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:33.349519 32402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:33.352396 32402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:33.352589 32402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:33.359211 32402 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34923
I20260621 08:00:33.359277 32469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34923 every 8 connection(s)
I20260621 08:00:33.360164 32402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:33.362771 32470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:33.366353 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 32402
I20260621 08:00:33.366505 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:33.366758 12158 external_mini_cluster.cc:1428] Setting key 2a0e840ddb1e0e13c3e24e09896b9c7b
I20260621 08:00:33.368387 32470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b: Bootstrap starting.
I20260621 08:00:33.370875 32470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:33.371706 32470 log.cc:824] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:33.373514 32470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b: No bootstrap required, opened a new log
Jun 21 08:00:33 dist-test-slave-0pn3 krb5kdc[32389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 08:00:33.377355 32470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260621 08:00:33.377579 32470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:33.377668 32470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ecf4a42f7bc41199c73f537c6ede43b, State: Initialized, Role: FOLLOWER
I20260621 08:00:33.378140 32470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260621 08:00:33.378293 32470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:33.378363 32470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:33.378460 32470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:33.379501 32470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260621 08:00:33.379887 32470 leader_election.cc:302] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ecf4a42f7bc41199c73f537c6ede43b; no voters:
I20260621 08:00:33.380247 32470 leader_election.cc:288] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:33.380338 32475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:33.380544 32475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [term 1 LEADER]: Becoming Leader. State: Replica: 4ecf4a42f7bc41199c73f537c6ede43b, State: Running, Role: LEADER
I20260621 08:00:33.380810 32475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } }
I20260621 08:00:33.381451 32470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:33.381902 32475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ecf4a42f7bc41199c73f537c6ede43b. Latest consensus state: current_term: 1 leader_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } } }
I20260621 08:00:33.382009 32475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:33.382265 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf4a42f7bc41199c73f537c6ede43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34923 } } }
I20260621 08:00:33.382371 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:33.382393 32479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:33.382332 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:33.368111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53813 (local address 127.0.0.1:34923)
0621 08:00:33.368507 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:33.368526 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:33.368566 (+ 40us) server_negotiation.cc:407] Connection header received
0621 08:00:33.369315 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:33.369335 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:33.369781 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:33.370107 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:33.370794 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:33.372232 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:33.373077 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:33.373714 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:33.373871 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:33.376644 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:33.376648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:33.376659 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:33.376684 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:33.378614 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:33.379139 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:33.379145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:33.379150 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:33.379287 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:33.379719 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:33.379722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:33.379724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:33.380219 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:33.380493 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:33.380961 (+ 468us) server_negotiation.cc:299] Negotiation successful
0621 08:00:33.381188 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":96,"threads_started":1}
I20260621 08:00:33.385568 32479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:33.385756 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:33.391276 32479 catalog_manager.cc:1347] Generated new cluster ID: 0073ef5c06744881a19f4e1c6f3ccdd1
I20260621 08:00:33.391377 32479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:33.396421 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:33.387658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38271 (local address 127.0.0.1:34923)
0621 08:00:33.387755 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:33.387759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:33.388199 (+ 440us) server_negotiation.cc:407] Connection header received
0621 08:00:33.388347 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:33.388350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:33.388398 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:33.388469 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:33.389148 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:33.389827 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:33.390846 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:33.391205 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:33.391305 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:33.393041 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:33.393045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:33.393047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:33.393071 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:33.394649 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:33.395172 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:33.395175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:33.395177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:33.395250 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:33.395602 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:33.395605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:33.395606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:33.395768 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:33.395905 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:33.396177 (+ 272us) server_negotiation.cc:299] Negotiation successful
0621 08:00:33.396257 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260621 08:00:33.415042 32479 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:33.416198 32479 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:33.425235 32479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ecf4a42f7bc41199c73f537c6ede43b: Generated new TSK 0
I20260621 08:00:33.425995 32479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:33.431309 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 32402
W20260621 08:00:33.436611 32387 connection.cc:535] client connection to 127.0.0.1:34923 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34923): Connection reset by peer (error 104)
2026-06-21T08:00:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:33 dist-test-slave-0pn3 krb5kdc[32501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:33 dist-test-slave-0pn3 krb5kdc[32501](info): set up 2 sockets
Jun 21 08:00:33 dist-test-slave-0pn3 krb5kdc[32501](info): commencing operation
krb5kdc: starting...
W20260621 08:00:34.448625 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:34 dist-test-slave-0pn3 krb5kdc[32501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:34.565404 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:34.670708 32514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:34.670996 32514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:34.671077 32514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:34.675053 32514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:34.675145 32514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:34.675190 32514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:34.675248 32514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:34.675288 32514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:34.680410 32514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:34.681394 32514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:34.682374 32514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:34.690840 32522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:34.690908 32520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:34.690992 32514 server_base.cc:1034] running on GCE node
W20260621 08:00:34.690909 32519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:34.691632 32514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:34.692817 32514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:34.694022 32514 hybrid_clock.cc:648] HybridClock initialized: now 1782028834694009 us; error 26 us; skew 500 ppm
Jun 21 08:00:34 dist-test-slave-0pn3 krb5kdc[32501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:34.696812 32514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:34.697649 32514 webserver.cc:460] Webserver started at http://127.0.0.1:39887/ using document root <none> and password file <none>
I20260621 08:00:34.698185 32514 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:34.698240 32514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:34.698397 32514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:34.700304 32514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "7f37835e61d94aa2b1047992869b8a10"
format_stamp: "Formatted at 2026-06-21 08:00:34 on dist-test-slave-0pn3"
server_key: "4b374d53878542da7103446c7c9082d3"
server_key_iv: "a5fc111d9686b02391d004881f1757db"
server_key_version: "encryptionkey@0"
I20260621 08:00:34.700937 32514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "7f37835e61d94aa2b1047992869b8a10"
format_stamp: "Formatted at 2026-06-21 08:00:34 on dist-test-slave-0pn3"
server_key: "4b374d53878542da7103446c7c9082d3"
server_key_iv: "a5fc111d9686b02391d004881f1757db"
server_key_version: "encryptionkey@0"
I20260621 08:00:34.704560 32514 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260621 08:00:34.706840 32529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:34.708019 32514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:34.708145 32514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "7f37835e61d94aa2b1047992869b8a10"
format_stamp: "Formatted at 2026-06-21 08:00:34 on dist-test-slave-0pn3"
server_key: "4b374d53878542da7103446c7c9082d3"
server_key_iv: "a5fc111d9686b02391d004881f1757db"
server_key_version: "encryptionkey@0"
I20260621 08:00:34.708244 32514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:34.732955 32514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:34.736259 32514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:34.736512 32514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:34.743593 32514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46231
I20260621 08:00:34.743613 32581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46231 every 8 connection(s)
I20260621 08:00:34.744571 32514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:34.747196 32582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:34.751530 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 32514
I20260621 08:00:34.751665 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:34.751929 12158 external_mini_cluster.cc:1428] Setting key 611d6779adaf68f05b296e4656baa8f9
I20260621 08:00:34.752810 32582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10: Bootstrap starting.
I20260621 08:00:34.755330 32582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:34.756165 32582 log.cc:824] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:34.758044 32582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10: No bootstrap required, opened a new log
I20260621 08:00:34.761643 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:34.753309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56619 (local address 127.0.0.1:46231)
0621 08:00:34.753726 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:34.753745 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:34.753791 (+ 46us) server_negotiation.cc:407] Connection header received
0621 08:00:34.754596 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:34.754617 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:34.755029 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:34.755377 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:34.755798 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:34.757132 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:34.757995 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:34.758653 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:34.758814 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:34.759637 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:34.759643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:34.759656 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:34.760140 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:34.760199 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:34.760225 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:34.760368 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:34.760574 (+ 206us) server_negotiation.cc:299] Negotiation successful
0621 08:00:34.760722 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":122,"threads_started":1}
I20260621 08:00:34.762228 32582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f37835e61d94aa2b1047992869b8a10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260621 08:00:34.762456 32582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:34.762523 32582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f37835e61d94aa2b1047992869b8a10, State: Initialized, Role: FOLLOWER
I20260621 08:00:34.762887 32582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f37835e61d94aa2b1047992869b8a10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260621 08:00:34.763029 32582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:34.763108 32582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:34.763224 32582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:34.764161 32582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f37835e61d94aa2b1047992869b8a10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260621 08:00:34.764479 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:34.764521 32582 leader_election.cc:302] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f37835e61d94aa2b1047992869b8a10; no voters:
I20260621 08:00:34.764844 32582 leader_election.cc:288] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:34.764952 32587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:34.765173 32587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [term 1 LEADER]: Becoming Leader. State: Replica: 7f37835e61d94aa2b1047992869b8a10, State: Running, Role: LEADER
I20260621 08:00:34.765484 32587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f37835e61d94aa2b1047992869b8a10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260621 08:00:34.766094 32582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:34.766631 32587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f37835e61d94aa2b1047992869b8a10. Latest consensus state: current_term: 1 leader_uuid: "7f37835e61d94aa2b1047992869b8a10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f37835e61d94aa2b1047992869b8a10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } } }
I20260621 08:00:34.766742 32587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:34.767294 32588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f37835e61d94aa2b1047992869b8a10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f37835e61d94aa2b1047992869b8a10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } } }
I20260621 08:00:34.767390 32588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:34.767678 32599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:34.770725 32599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:34.771873 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:34.766669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40795 (local address 127.0.0.1:46231)
0621 08:00:34.766766 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:34.766771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:34.767330 (+ 559us) server_negotiation.cc:407] Connection header received
0621 08:00:34.767477 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:34.767481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:34.767538 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:34.767610 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:34.767988 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:34.768744 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:34.769870 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:34.770223 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:34.770319 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:34.771403 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:34.771407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:34.771410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:34.771422 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:34.771446 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:34.771459 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:34.771512 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:34.771710 (+ 198us) server_negotiation.cc:299] Negotiation successful
0621 08:00:34.771742 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":34}
I20260621 08:00:34.776077 32599 catalog_manager.cc:1347] Generated new cluster ID: 4b117b5552d547149f0510a99576dfa2
I20260621 08:00:34.776146 32599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:34.807349 32599 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:34.808647 32599 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:34.812813 32599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f37835e61d94aa2b1047992869b8a10: Generated new TSK 0
I20260621 08:00:34.813601 32599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:34.820503 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 32514
W20260621 08:00:34.826120 32499 connection.cc:535] client connection to 127.0.0.1:46231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46231): Connection reset by peer (error 104)
2026-06-21T08:00:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:34 dist-test-slave-0pn3 krb5kdc[32613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:34 dist-test-slave-0pn3 krb5kdc[32613](info): set up 2 sockets
Jun 21 08:00:34 dist-test-slave-0pn3 krb5kdc[32613](info): commencing operation
krb5kdc: starting...
W20260621 08:00:35.848644 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:35 dist-test-slave-0pn3 krb5kdc[32613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:35.965333 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:36.071537 32626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:36.071789 32626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:36.071871 32626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:36.075805 32626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:36.075860 32626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:36.075879 32626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:36.075912 32626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:36.075927 32626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:36.080890 32626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:36.081869 32626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:36.082861 32626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:36.090909 32634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:36.090973 32632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:36.091111 32631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:36.091172 32626 server_base.cc:1034] running on GCE node
I20260621 08:00:36.091673 32626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:36.092684 32626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:36.093838 32626 hybrid_clock.cc:648] HybridClock initialized: now 1782028836093812 us; error 38 us; skew 500 ppm
Jun 21 08:00:36 dist-test-slave-0pn3 krb5kdc[32613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:36.096436 32626 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:36.097214 32626 webserver.cc:460] Webserver started at http://127.0.0.1:36835/ using document root <none> and password file <none>
I20260621 08:00:36.097702 32626 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:36.097756 32626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:36.097970 32626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:36.099921 32626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "1bcb49c648f24411a997e71185ed3d59"
format_stamp: "Formatted at 2026-06-21 08:00:36 on dist-test-slave-0pn3"
server_key: "9c88c0168e4b2b27c3a63cb5b90922a8"
server_key_iv: "ae8cc838a1cc9093dd70012457c694e1"
server_key_version: "encryptionkey@0"
I20260621 08:00:36.100522 32626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "1bcb49c648f24411a997e71185ed3d59"
format_stamp: "Formatted at 2026-06-21 08:00:36 on dist-test-slave-0pn3"
server_key: "9c88c0168e4b2b27c3a63cb5b90922a8"
server_key_iv: "ae8cc838a1cc9093dd70012457c694e1"
server_key_version: "encryptionkey@0"
I20260621 08:00:36.103955 32626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260621 08:00:36.106489 32641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:36.107657 32626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:36.107793 32626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "1bcb49c648f24411a997e71185ed3d59"
format_stamp: "Formatted at 2026-06-21 08:00:36 on dist-test-slave-0pn3"
server_key: "9c88c0168e4b2b27c3a63cb5b90922a8"
server_key_iv: "ae8cc838a1cc9093dd70012457c694e1"
server_key_version: "encryptionkey@0"
I20260621 08:00:36.107900 32626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:36.112695 32626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:36.115564 32626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:36.115761 32626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:36.122378 32626 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32783
I20260621 08:00:36.122432 32693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32783 every 8 connection(s)
I20260621 08:00:36.123225 32626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:36.125844 32694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:36.130345 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 32626
I20260621 08:00:36.130489 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:36.130761 12158 external_mini_cluster.cc:1428] Setting key b6a2ea3ca461010de98c169f93230882
I20260621 08:00:36.131064 32694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59: Bootstrap starting.
I20260621 08:00:36.133365 32694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:36.134148 32694 log.cc:824] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:36.135659 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:36.132023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55677 (local address 127.0.0.1:32783)
0621 08:00:36.132499 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:36.132517 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:36.132550 (+ 33us) server_negotiation.cc:407] Connection header received
0621 08:00:36.133122 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:36.133136 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:36.133389 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:36.133743 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:36.134534 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:36.134546 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:36.134557 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:36.134567 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:36.134636 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:36.134655 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:36.134768 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:36.134874 (+ 106us) server_negotiation.cc:299] Negotiation successful
0621 08:00:36.135033 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":128,"threads_started":1}
I20260621 08:00:36.136039 32694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59: No bootstrap required, opened a new log
I20260621 08:00:36.138515 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:36.140128 32694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcb49c648f24411a997e71185ed3d59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260621 08:00:36.140432 32694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:36.140517 32694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcb49c648f24411a997e71185ed3d59, State: Initialized, Role: FOLLOWER
I20260621 08:00:36.140887 32694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcb49c648f24411a997e71185ed3d59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260621 08:00:36.140998 32694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:36.141042 32694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:36.141108 32694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:36.141886 32694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcb49c648f24411a997e71185ed3d59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260621 08:00:36.142163 32694 leader_election.cc:302] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bcb49c648f24411a997e71185ed3d59; no voters:
I20260621 08:00:36.142385 32694 leader_election.cc:288] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:36.142516 32704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:36.142733 32704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcb49c648f24411a997e71185ed3d59, State: Running, Role: LEADER
I20260621 08:00:36.143072 32704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcb49c648f24411a997e71185ed3d59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260621 08:00:36.143589 32694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:36.144775 32706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bcb49c648f24411a997e71185ed3d59. Latest consensus state: current_term: 1 leader_uuid: "1bcb49c648f24411a997e71185ed3d59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcb49c648f24411a997e71185ed3d59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } } }
I20260621 08:00:36.144908 32706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:36.145226 32705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bcb49c648f24411a997e71185ed3d59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcb49c648f24411a997e71185ed3d59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } } }
I20260621 08:00:36.145337 32705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:36.145359 32713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:36.147727 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:36.140422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47853 (local address 127.0.0.1:32783)
0621 08:00:36.140540 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:36.140544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:36.146264 (+ 5720us) server_negotiation.cc:407] Connection header received
0621 08:00:36.146428 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:36.146431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:36.146470 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:36.146573 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:36.147324 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:36.147327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:36.147330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:36.147341 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:36.147375 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:36.147387 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:36.147480 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:36.147569 (+ 89us) server_negotiation.cc:299] Negotiation successful
0621 08:00:36.147607 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 08:00:36.148739 32713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:36.154598 32713 catalog_manager.cc:1347] Generated new cluster ID: c8236d533d13436786ee023d2148ff50
I20260621 08:00:36.154662 32713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:36.186358 32713 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:36.186743 32713 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:36.199249 32713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bcb49c648f24411a997e71185ed3d59: Generated new TSK 0
I20260621 08:00:36.200040 32713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:36.202924 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 32626
2026-06-21T08:00:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:36 dist-test-slave-0pn3 krb5kdc[32725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:36 dist-test-slave-0pn3 krb5kdc[32725](info): set up 2 sockets
Jun 21 08:00:36 dist-test-slave-0pn3 krb5kdc[32725](info): commencing operation
krb5kdc: starting...
W20260621 08:00:37.232625 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:37 dist-test-slave-0pn3 krb5kdc[32725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:37.346880 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:37.452675 32738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:37.452920 32738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:37.452966 32738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:37.456844 32738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:37.456889 32738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:37.456907 32738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:37.456923 32738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:37.456976 32738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:37.462035 32738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:37.463253 32738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:37.464452 32738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:37.472586 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:37.472643 32746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:37.472693 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:37.472746 32738 server_base.cc:1034] running on GCE node
I20260621 08:00:37.473361 32738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:37.474488 32738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:37.475675 32738 hybrid_clock.cc:648] HybridClock initialized: now 1782028837475664 us; error 24 us; skew 500 ppm
Jun 21 08:00:37 dist-test-slave-0pn3 krb5kdc[32725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:37.481877 32738 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:37.482730 32738 webserver.cc:460] Webserver started at http://127.0.0.1:36805/ using document root <none> and password file <none>
I20260621 08:00:37.483242 32738 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:37.483327 32738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:37.483558 32738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:37.485551 32738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "cad60117fe3e45e087e42d769d76406b"
format_stamp: "Formatted at 2026-06-21 08:00:37 on dist-test-slave-0pn3"
server_key: "300882aab08001a3ab549b9f9c8c3456"
server_key_iv: "f1fb4c3fbd56055f141706d78420e8eb"
server_key_version: "encryptionkey@0"
I20260621 08:00:37.486104 32738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "cad60117fe3e45e087e42d769d76406b"
format_stamp: "Formatted at 2026-06-21 08:00:37 on dist-test-slave-0pn3"
server_key: "300882aab08001a3ab549b9f9c8c3456"
server_key_iv: "f1fb4c3fbd56055f141706d78420e8eb"
server_key_version: "encryptionkey@0"
I20260621 08:00:37.489488 32738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:37.491588 32753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:37.492736 32738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:37.492867 32738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "cad60117fe3e45e087e42d769d76406b"
format_stamp: "Formatted at 2026-06-21 08:00:37 on dist-test-slave-0pn3"
server_key: "300882aab08001a3ab549b9f9c8c3456"
server_key_iv: "f1fb4c3fbd56055f141706d78420e8eb"
server_key_version: "encryptionkey@0"
I20260621 08:00:37.492974 32738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:37.506589 32738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:37.509460 32738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:37.509665 32738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:37.516610 32738 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40131
I20260621 08:00:37.516638 337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40131 every 8 connection(s)
I20260621 08:00:37.517510 32738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:37.520023 338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:37.522845 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 32738
I20260621 08:00:37.522984 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:37.523222 12158 external_mini_cluster.cc:1428] Setting key 1a22a8809aaa2b89817eb1b5b6a61e7c
I20260621 08:00:37.525779 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b: Bootstrap starting.
I20260621 08:00:37.528245 338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:37.529014 338 log.cc:824] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:37.530850 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b: No bootstrap required, opened a new log
Jun 21 08:00:37 dist-test-slave-0pn3 krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 08:00:37.535117 338 raft_consensus.cc:348] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cad60117fe3e45e087e42d769d76406b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260621 08:00:37.535373 338 raft_consensus.cc:374] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:37.535439 338 raft_consensus.cc:729] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad60117fe3e45e087e42d769d76406b, State: Initialized, Role: FOLLOWER
I20260621 08:00:37.535900 338 consensus_queue.cc:260] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cad60117fe3e45e087e42d769d76406b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260621 08:00:37.536072 338 raft_consensus.cc:388] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:37.536150 338 raft_consensus.cc:482] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:37.536257 338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:37.537256 338 raft_consensus.cc:504] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cad60117fe3e45e087e42d769d76406b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260621 08:00:37.537612 338 leader_election.cc:302] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cad60117fe3e45e087e42d769d76406b; no voters:
I20260621 08:00:37.537930 338 leader_election.cc:288] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:37.537942 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:37.524567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53709 (local address 127.0.0.1:40131)
0621 08:00:37.524990 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:37.525009 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:37.525048 (+ 39us) server_negotiation.cc:407] Connection header received
0621 08:00:37.525804 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:37.525826 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:37.526228 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:37.526562 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:37.527194 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:37.528616 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:37.529498 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:37.530068 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:37.530211 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:37.532749 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:37.532755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:37.532781 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:37.532807 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:37.534721 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:37.535147 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:37.535153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:37.535158 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:37.535286 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:37.535591 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:37.535594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:37.535596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:37.536067 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:37.536348 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:37.536705 (+ 357us) server_negotiation.cc:299] Negotiation successful
0621 08:00:37.536993 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":103,"threads_started":1}
I20260621 08:00:37.537983 343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:37.538228 343 raft_consensus.cc:686] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [term 1 LEADER]: Becoming Leader. State: Replica: cad60117fe3e45e087e42d769d76406b, State: Running, Role: LEADER
I20260621 08:00:37.538609 343 consensus_queue.cc:237] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cad60117fe3e45e087e42d769d76406b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260621 08:00:37.539076 338 sys_catalog.cc:564] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:37.540676 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cad60117fe3e45e087e42d769d76406b. Latest consensus state: current_term: 1 leader_uuid: "cad60117fe3e45e087e42d769d76406b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cad60117fe3e45e087e42d769d76406b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } } }
I20260621 08:00:37.540772 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:37.541083 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:37.540988 345 sys_catalog.cc:455] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cad60117fe3e45e087e42d769d76406b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cad60117fe3e45e087e42d769d76406b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } } }
I20260621 08:00:37.541350 345 sys_catalog.cc:458] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:37.544225 357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:37.544304 357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:37.544474 358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:37.545302 358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:37.551223 358 catalog_manager.cc:1347] Generated new cluster ID: 7fa30a20ab26475681f9b3b5fb4922c8
I20260621 08:00:37.551345 358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:37.552717 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:37.542847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52731 (local address 127.0.0.1:40131)
0621 08:00:37.542954 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:37.542958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:37.543209 (+ 251us) server_negotiation.cc:407] Connection header received
0621 08:00:37.543342 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:37.543346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:37.543391 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:37.543478 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:37.544097 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:37.544849 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:37.545981 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:37.546303 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:37.546417 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:37.549184 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:37.549188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:37.549193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:37.549219 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:37.550726 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:37.551324 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:37.551329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:37.551331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:37.551465 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:37.551775 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:37.551778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:37.551780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:37.551994 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:37.552128 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:37.552410 (+ 282us) server_negotiation.cc:299] Negotiation successful
0621 08:00:37.552534 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 08:00:37.569334 358 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:37.570163 358 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:37.579739 358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cad60117fe3e45e087e42d769d76406b: Generated new TSK 0
I20260621 08:00:37.580453 358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:37.584436 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 32738
2026-06-21T08:00:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:37 dist-test-slave-0pn3 krb5kdc[369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:37 dist-test-slave-0pn3 krb5kdc[369](info): set up 2 sockets
Jun 21 08:00:37 dist-test-slave-0pn3 krb5kdc[369](info): commencing operation
krb5kdc: starting...
W20260621 08:00:38.608763 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:38 dist-test-slave-0pn3 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:38.725620 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:38.833618 382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:38.833961 382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:38.834055 382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:38.838150 382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:38.838270 382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:38.838310 382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:38.838372 382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:38.838411 382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:38.843784 382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:38.844976 382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:38.846028 382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:38.854460 388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:38.854449 387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:38.854568 382 server_base.cc:1034] running on GCE node
W20260621 08:00:38.854449 390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:38.855290 382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:38.856392 382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:38.857594 382 hybrid_clock.cc:648] HybridClock initialized: now 1782028838857567 us; error 35 us; skew 500 ppm
Jun 21 08:00:38 dist-test-slave-0pn3 krb5kdc[369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:38.860195 382 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:38.861157 382 webserver.cc:460] Webserver started at http://127.0.0.1:36983/ using document root <none> and password file <none>
I20260621 08:00:38.861691 382 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:38.861778 382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:38.861969 382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:38.863920 382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "7c48309da0a04dc08e9b99302ba0fc0c"
format_stamp: "Formatted at 2026-06-21 08:00:38 on dist-test-slave-0pn3"
server_key: "5e200632ea30ea5ff668ccc8fb674335"
server_key_iv: "eaef221138874f86e6aaa3ba5ed896bc"
server_key_version: "encryptionkey@0"
I20260621 08:00:38.864544 382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "7c48309da0a04dc08e9b99302ba0fc0c"
format_stamp: "Formatted at 2026-06-21 08:00:38 on dist-test-slave-0pn3"
server_key: "5e200632ea30ea5ff668ccc8fb674335"
server_key_iv: "eaef221138874f86e6aaa3ba5ed896bc"
server_key_version: "encryptionkey@0"
I20260621 08:00:38.868093 382 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 08:00:38.870611 397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:38.871892 382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:38.872033 382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "7c48309da0a04dc08e9b99302ba0fc0c"
format_stamp: "Formatted at 2026-06-21 08:00:38 on dist-test-slave-0pn3"
server_key: "5e200632ea30ea5ff668ccc8fb674335"
server_key_iv: "eaef221138874f86e6aaa3ba5ed896bc"
server_key_version: "encryptionkey@0"
I20260621 08:00:38.872152 382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:38.891918 382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:38.894754 382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:38.894956 382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:38.901916 382 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43557
I20260621 08:00:38.901942 449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43557 every 8 connection(s)
I20260621 08:00:38.902854 382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:38.905597 450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:38.911024 450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c: Bootstrap starting.
I20260621 08:00:38.911532 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 382
I20260621 08:00:38.911674 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:38.911944 12158 external_mini_cluster.cc:1428] Setting key 740a2c18c01ac075dc42e6e2d14d691f
I20260621 08:00:38.913818 450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:38.914561 450 log.cc:824] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:38.916640 450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c: No bootstrap required, opened a new log
I20260621 08:00:38.920837 450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43557 } }
I20260621 08:00:38.921062 450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:38.921139 450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c48309da0a04dc08e9b99302ba0fc0c, State: Initialized, Role: FOLLOWER
I20260621 08:00:38.921168 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:38.913287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36679 (local address 127.0.0.1:43557)
0621 08:00:38.913730 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:38.913745 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:38.913781 (+ 36us) server_negotiation.cc:407] Connection header received
0621 08:00:38.914469 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:38.914486 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:38.914907 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:38.915282 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:38.915646 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:38.916673 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:38.917936 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:38.918399 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:38.918512 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:38.919381 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:38.919386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:38.919397 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:38.919756 (+ 359us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:38.919834 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:38.919870 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:38.919981 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:38.920195 (+ 214us) server_negotiation.cc:299] Negotiation successful
0621 08:00:38.920438 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260621 08:00:38.921609 450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43557 } }
I20260621 08:00:38.921761 450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:38.921820 450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:38.921907 450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:38.922780 450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43557 } }
I20260621 08:00:38.923120 450 leader_election.cc:302] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c48309da0a04dc08e9b99302ba0fc0c; no voters:
I20260621 08:00:38.923456 450 leader_election.cc:288] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:38.923801 456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:38.923878 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:38.923990 456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [term 1 LEADER]: Becoming Leader. State: Replica: 7c48309da0a04dc08e9b99302ba0fc0c, State: Running, Role: LEADER
I20260621 08:00:38.924247 456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43557 } }
I20260621 08:00:38.924620 450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:38.925516 456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c48309da0a04dc08e9b99302ba0fc0c. Latest consensus state: current_term: 1 leader_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43557 } } }
I20260621 08:00:38.925622 456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:38.925925 455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c48309da0a04dc08e9b99302ba0fc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43557 } } }
I20260621 08:00:38.926020 455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:38.929683 475 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:38.929764 475 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:38.929823 468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:38.930703 468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:38.931033 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:38.926392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40479 (local address 127.0.0.1:43557)
0621 08:00:38.926481 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:38.926485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:38.926695 (+ 210us) server_negotiation.cc:407] Connection header received
0621 08:00:38.926863 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:38.926867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:38.926916 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:38.926973 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:38.927356 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:38.928099 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:38.929211 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:38.929540 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:38.929649 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:38.930512 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:38.930516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:38.930519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:38.930533 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:38.930563 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:38.930577 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:38.930658 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:38.930843 (+ 185us) server_negotiation.cc:299] Negotiation successful
0621 08:00:38.930907 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260621 08:00:38.936088 468 catalog_manager.cc:1347] Generated new cluster ID: 5ebdaca9f3a64b7e98d9da81673a0c1d
I20260621 08:00:38.936148 468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:38.949589 468 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:38.950939 468 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:38.966897 468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c48309da0a04dc08e9b99302ba0fc0c: Generated new TSK 0
I20260621 08:00:38.967851 468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:38.977402 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 382
2026-06-21T08:00:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:39 dist-test-slave-0pn3 krb5kdc[481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:39 dist-test-slave-0pn3 krb5kdc[481](info): set up 2 sockets
Jun 21 08:00:39 dist-test-slave-0pn3 krb5kdc[481](info): commencing operation
krb5kdc: starting...
W20260621 08:00:40.008738 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:40 dist-test-slave-0pn3 krb5kdc[481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:40.127332 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:40.232152 494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:40.232493 494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:40.232581 494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:40.236496 494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:40.236552 494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:40.236572 494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:40.236627 494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:40.236660 494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:40.241811 494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:40.242820 494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:40.243793 494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:40.252002 502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:40.252106 500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:40.252099 499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:40.252251 494 server_base.cc:1034] running on GCE node
I20260621 08:00:40.252846 494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:40.253891 494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:40.255046 494 hybrid_clock.cc:648] HybridClock initialized: now 1782028840255016 us; error 42 us; skew 500 ppm
Jun 21 08:00:40 dist-test-slave-0pn3 krb5kdc[481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:40.257647 494 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:40.258528 494 webserver.cc:460] Webserver started at http://127.0.0.1:38859/ using document root <none> and password file <none>
I20260621 08:00:40.259071 494 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:40.259155 494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:40.259377 494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:40.261345 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "347b5496fd9c4eeaab423e6e29aff402"
format_stamp: "Formatted at 2026-06-21 08:00:40 on dist-test-slave-0pn3"
server_key: "c73854013105bbd51d7f889f4168a088"
server_key_iv: "506a2614fa03322cb12f9a51172ebc1d"
server_key_version: "encryptionkey@0"
I20260621 08:00:40.261870 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "347b5496fd9c4eeaab423e6e29aff402"
format_stamp: "Formatted at 2026-06-21 08:00:40 on dist-test-slave-0pn3"
server_key: "c73854013105bbd51d7f889f4168a088"
server_key_iv: "506a2614fa03322cb12f9a51172ebc1d"
server_key_version: "encryptionkey@0"
I20260621 08:00:40.265354 494 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260621 08:00:40.267642 509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:40.268846 494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:40.269001 494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "347b5496fd9c4eeaab423e6e29aff402"
format_stamp: "Formatted at 2026-06-21 08:00:40 on dist-test-slave-0pn3"
server_key: "c73854013105bbd51d7f889f4168a088"
server_key_iv: "506a2614fa03322cb12f9a51172ebc1d"
server_key_version: "encryptionkey@0"
I20260621 08:00:40.269110 494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:40.273561 494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:40.276367 494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:40.276539 494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:40.283288 494 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46785
I20260621 08:00:40.283308 561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46785 every 8 connection(s)
I20260621 08:00:40.284132 494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:40.286788 562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:40.291950 562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402: Bootstrap starting.
I20260621 08:00:40.292467 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 494
I20260621 08:00:40.292608 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:40.292892 12158 external_mini_cluster.cc:1428] Setting key ed127e2b1b2f91ff3755a2b56b428aa2
I20260621 08:00:40.294662 562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:40.295392 562 log.cc:824] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:40.297640 562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402: No bootstrap required, opened a new log
I20260621 08:00:40.298050 565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:40.294142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40481 (local address 127.0.0.1:46785)
0621 08:00:40.294572 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:40.294585 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:40.294620 (+ 35us) server_negotiation.cc:407] Connection header received
0621 08:00:40.295273 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:40.295290 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:40.295607 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:40.295973 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:40.296782 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:40.296787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:40.296797 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:40.296805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:40.296850 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:40.296867 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:40.296947 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:40.297144 (+ 197us) server_negotiation.cc:299] Negotiation successful
0621 08:00:40.297317 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":133,"threads_started":1}
I20260621 08:00:40.300807 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:40.301550 562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347b5496fd9c4eeaab423e6e29aff402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46785 } }
I20260621 08:00:40.301788 562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:40.301848 562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347b5496fd9c4eeaab423e6e29aff402, State: Initialized, Role: FOLLOWER
I20260621 08:00:40.302302 562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347b5496fd9c4eeaab423e6e29aff402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46785 } }
I20260621 08:00:40.302420 562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:40.302495 562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:40.302588 562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:40.303529 562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347b5496fd9c4eeaab423e6e29aff402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46785 } }
I20260621 08:00:40.303870 562 leader_election.cc:302] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347b5496fd9c4eeaab423e6e29aff402; no voters:
I20260621 08:00:40.304194 562 leader_election.cc:288] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:40.304347 572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:40.304551 572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [term 1 LEADER]: Becoming Leader. State: Replica: 347b5496fd9c4eeaab423e6e29aff402, State: Running, Role: LEADER
I20260621 08:00:40.304579 565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:40.302833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37225 (local address 127.0.0.1:46785)
0621 08:00:40.302951 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:40.302964 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:40.303147 (+ 183us) server_negotiation.cc:407] Connection header received
0621 08:00:40.303281 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:40.303288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:40.303317 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:40.303408 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:40.304143 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:40.304147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:40.304149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:40.304159 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:40.304185 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:40.304204 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:40.304289 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:40.304401 (+ 112us) server_negotiation.cc:299] Negotiation successful
0621 08:00:40.304439 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 08:00:40.304878 572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347b5496fd9c4eeaab423e6e29aff402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46785 } }
I20260621 08:00:40.305507 562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:40.306044 572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347b5496fd9c4eeaab423e6e29aff402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347b5496fd9c4eeaab423e6e29aff402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46785 } } }
I20260621 08:00:40.306169 572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:40.306380 572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347b5496fd9c4eeaab423e6e29aff402. Latest consensus state: current_term: 1 leader_uuid: "347b5496fd9c4eeaab423e6e29aff402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347b5496fd9c4eeaab423e6e29aff402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46785 } } }
I20260621 08:00:40.306476 572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:40.306669 578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:40.309777 578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:40.314731 578 catalog_manager.cc:1347] Generated new cluster ID: 3fe1500c657d43598fb9420711480100
I20260621 08:00:40.314790 578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:40.321748 578 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:40.322057 578 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:40.330454 578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347b5496fd9c4eeaab423e6e29aff402: Generated new TSK 0
I20260621 08:00:40.331305 578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:40.340513 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 494
2026-06-21T08:00:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:40 dist-test-slave-0pn3 krb5kdc[593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:40 dist-test-slave-0pn3 krb5kdc[593](info): set up 2 sockets
Jun 21 08:00:40 dist-test-slave-0pn3 krb5kdc[593](info): commencing operation
krb5kdc: starting...
W20260621 08:00:41.372776 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:41 dist-test-slave-0pn3 krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:41.489614 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:41.596906 606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:41.597218 606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:41.597289 606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:41.601238 606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:41.601331 606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:41.601370 606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:41.601429 606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:41.601480 606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:41.606669 606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:41.607738 606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:41.608768 606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:41.619261 611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:41.619656 614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:41.619752 612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:41.620146 606 server_base.cc:1034] running on GCE node
I20260621 08:00:41.620932 606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:41.622025 606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:41.623212 606 hybrid_clock.cc:648] HybridClock initialized: now 1782028841623197 us; error 27 us; skew 500 ppm
Jun 21 08:00:41 dist-test-slave-0pn3 krb5kdc[593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:41.625998 606 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:41.626951 606 webserver.cc:460] Webserver started at http://127.0.0.1:41967/ using document root <none> and password file <none>
I20260621 08:00:41.627522 606 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:41.627614 606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:41.627859 606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:41.629961 606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "bb961feeeb414e0eb172cc930d7e510c"
format_stamp: "Formatted at 2026-06-21 08:00:41 on dist-test-slave-0pn3"
server_key: "fd502a546a52ac01c7b253f0a35cd02f"
server_key_iv: "3aed3310a6bac23553bac600f4ce8f89"
server_key_version: "encryptionkey@0"
I20260621 08:00:41.630539 606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "bb961feeeb414e0eb172cc930d7e510c"
format_stamp: "Formatted at 2026-06-21 08:00:41 on dist-test-slave-0pn3"
server_key: "fd502a546a52ac01c7b253f0a35cd02f"
server_key_iv: "3aed3310a6bac23553bac600f4ce8f89"
server_key_version: "encryptionkey@0"
I20260621 08:00:41.634315 606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 08:00:41.636720 621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:41.637982 606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:41.638129 606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "bb961feeeb414e0eb172cc930d7e510c"
format_stamp: "Formatted at 2026-06-21 08:00:41 on dist-test-slave-0pn3"
server_key: "fd502a546a52ac01c7b253f0a35cd02f"
server_key_iv: "3aed3310a6bac23553bac600f4ce8f89"
server_key_version: "encryptionkey@0"
I20260621 08:00:41.638242 606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:41.655781 606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:41.658659 606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:41.658885 606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:41.665691 606 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40203
I20260621 08:00:41.665721 673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40203 every 8 connection(s)
I20260621 08:00:41.666617 606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:41.669349 674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:41.674800 674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c: Bootstrap starting.
I20260621 08:00:41.675969 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 606
I20260621 08:00:41.676126 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:41.676429 12158 external_mini_cluster.cc:1428] Setting key d77a007e4078862bed9879da8976fa05
I20260621 08:00:41.677625 674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:41.678323 674 log.cc:824] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:41.680473 674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c: No bootstrap required, opened a new log
I20260621 08:00:41.684864 674 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb961feeeb414e0eb172cc930d7e510c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260621 08:00:41.685154 674 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:41.685214 674 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb961feeeb414e0eb172cc930d7e510c, State: Initialized, Role: FOLLOWER
Jun 21 08:00:41 dist-test-slave-0pn3 krb5kdc[593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 08:00:41.685660 674 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb961feeeb414e0eb172cc930d7e510c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260621 08:00:41.685827 674 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:41.685891 674 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:41.685985 674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:41.686863 674 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb961feeeb414e0eb172cc930d7e510c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260621 08:00:41.687202 674 leader_election.cc:302] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb961feeeb414e0eb172cc930d7e510c; no voters:
I20260621 08:00:41.687518 674 leader_election.cc:288] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:41.687623 679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:41.687848 679 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [term 1 LEADER]: Becoming Leader. State: Replica: bb961feeeb414e0eb172cc930d7e510c, State: Running, Role: LEADER
I20260621 08:00:41.688211 679 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb961feeeb414e0eb172cc930d7e510c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } }
I20260621 08:00:41.688861 674 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:41.689534 681 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb961feeeb414e0eb172cc930d7e510c. Latest consensus state: current_term: 1 leader_uuid: "bb961feeeb414e0eb172cc930d7e510c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb961feeeb414e0eb172cc930d7e510c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } } }
I20260621 08:00:41.689683 681 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:41.690042 680 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb961feeeb414e0eb172cc930d7e510c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb961feeeb414e0eb172cc930d7e510c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40203 } } }
I20260621 08:00:41.690196 680 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:41.692533 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:41.677580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:45751 (local address 127.0.0.1:40203)
0621 08:00:41.678075 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:41.678090 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:41.678123 (+ 33us) server_negotiation.cc:407] Connection header received
0621 08:00:41.678801 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:41.678822 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:41.679156 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:41.679431 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:41.680372 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:41.681380 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:41.682521 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:41.683069 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:41.683183 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:41.686321 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:41.686326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:41.686339 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:41.686367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:41.688506 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:41.689079 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:41.689085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:41.689090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:41.689220 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:41.689620 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:41.689626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:41.689631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:41.690218 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:41.690426 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:41.691438 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0621 08:00:41.691669 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":173,"threads_started":1}
W20260621 08:00:41.695039 694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:41.695117 694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:41.695183 685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:41.695401 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:41.696025 685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:41.701658 685 catalog_manager.cc:1347] Generated new cluster ID: 0fd3a4bde5e34bb48fa87c3a91a70f1f
I20260621 08:00:41.701721 685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:41.709230 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:41.697948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:37667 (local address 127.0.0.1:40203)
0621 08:00:41.698059 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:41.698063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:41.698398 (+ 335us) server_negotiation.cc:407] Connection header received
0621 08:00:41.698538 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:41.698541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:41.698588 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:41.698668 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:41.699494 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:41.700130 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:41.701332 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:41.701655 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:41.701763 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:41.705893 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:41.705896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:41.705899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:41.705926 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:41.707395 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:41.707878 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:41.707882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:41.707883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:41.707963 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:41.708286 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:41.708289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:41.708291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:41.708537 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:41.708705 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:41.708962 (+ 257us) server_negotiation.cc:299] Negotiation successful
0621 08:00:41.709057 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":43}
I20260621 08:00:41.711839 685 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:41.713105 685 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:41.730983 685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb961feeeb414e0eb172cc930d7e510c: Generated new TSK 0
I20260621 08:00:41.731716 685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:41.743315 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 606
W20260621 08:00:41.748427 591 connection.cc:535] client connection to 127.0.0.1:40203 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40203): Connection reset by peer (error 104)
2026-06-21T08:00:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:41 dist-test-slave-0pn3 krb5kdc[705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:41 dist-test-slave-0pn3 krb5kdc[705](info): set up 2 sockets
Jun 21 08:00:41 dist-test-slave-0pn3 krb5kdc[705](info): commencing operation
krb5kdc: starting...
W20260621 08:00:42.768816 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:42 dist-test-slave-0pn3 krb5kdc[705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:42.885437 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:42.995056 718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:42.995329 718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:42.995373 718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:42.999449 718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:42.999513 718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:42.999533 718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:42.999550 718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:42.999607 718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:43.004956 718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:43.006007 718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:43.007048 718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:43.015375 723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:43.015420 726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:43.015492 718 server_base.cc:1034] running on GCE node
W20260621 08:00:43.015394 724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:43.016285 718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:43.017385 718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:43.018591 718 hybrid_clock.cc:648] HybridClock initialized: now 1782028843018561 us; error 41 us; skew 500 ppm
Jun 21 08:00:43 dist-test-slave-0pn3 krb5kdc[705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:43.025957 718 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:43.026886 718 webserver.cc:460] Webserver started at http://127.0.0.1:45285/ using document root <none> and password file <none>
I20260621 08:00:43.027462 718 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:43.027524 718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:43.027765 718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:43.029896 718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "e8dd9e34a6d94a739b7482783eb4d794"
format_stamp: "Formatted at 2026-06-21 08:00:43 on dist-test-slave-0pn3"
server_key: "8a1d64ebb588b5d5a18064e3965da662"
server_key_iv: "355b253fa2fd32b16d0bedfbfe1114b9"
server_key_version: "encryptionkey@0"
I20260621 08:00:43.030452 718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "e8dd9e34a6d94a739b7482783eb4d794"
format_stamp: "Formatted at 2026-06-21 08:00:43 on dist-test-slave-0pn3"
server_key: "8a1d64ebb588b5d5a18064e3965da662"
server_key_iv: "355b253fa2fd32b16d0bedfbfe1114b9"
server_key_version: "encryptionkey@0"
I20260621 08:00:43.034291 718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260621 08:00:43.036710 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:43.037904 718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:43.038030 718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "e8dd9e34a6d94a739b7482783eb4d794"
format_stamp: "Formatted at 2026-06-21 08:00:43 on dist-test-slave-0pn3"
server_key: "8a1d64ebb588b5d5a18064e3965da662"
server_key_iv: "355b253fa2fd32b16d0bedfbfe1114b9"
server_key_version: "encryptionkey@0"
I20260621 08:00:43.038130 718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:43.051918 718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:43.054574 718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:43.054791 718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:43.061733 718 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42299
I20260621 08:00:43.061739 785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42299 every 8 connection(s)
I20260621 08:00:43.062690 718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:43.065523 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:43.070678 786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794: Bootstrap starting.
I20260621 08:00:43.071540 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 718
I20260621 08:00:43.071717 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:43.072005 12158 external_mini_cluster.cc:1428] Setting key a0374ec19fa29fff8baa4ec9bc778c48
I20260621 08:00:43.073462 786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:43.074179 786 log.cc:824] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:43.076146 786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794: No bootstrap required, opened a new log
I20260621 08:00:43.080487 786 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8dd9e34a6d94a739b7482783eb4d794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260621 08:00:43.080713 786 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:43.080791 786 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8dd9e34a6d94a739b7482783eb4d794, State: Initialized, Role: FOLLOWER
I20260621 08:00:43.080922 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:43.073090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:47185 (local address 127.0.0.1:42299)
0621 08:00:43.073530 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:43.073545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:43.073605 (+ 60us) server_negotiation.cc:407] Connection header received
0621 08:00:43.074233 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:43.074249 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:43.074697 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:43.075045 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:43.075369 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:43.076413 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:43.077561 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:43.078024 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:43.078142 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:43.079127 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:43.079132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:43.079142 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:43.079518 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:43.079570 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:43.079591 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:43.079703 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:43.079945 (+ 242us) server_negotiation.cc:299] Negotiation successful
0621 08:00:43.080083 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":155,"threads_started":1}
I20260621 08:00:43.081287 786 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8dd9e34a6d94a739b7482783eb4d794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260621 08:00:43.081437 786 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:43.081499 786 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:43.081588 786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:43.082470 786 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8dd9e34a6d94a739b7482783eb4d794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260621 08:00:43.082829 786 leader_election.cc:302] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8dd9e34a6d94a739b7482783eb4d794; no voters:
I20260621 08:00:43.083177 786 leader_election.cc:288] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:43.083304 791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:43.083505 791 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [term 1 LEADER]: Becoming Leader. State: Replica: e8dd9e34a6d94a739b7482783eb4d794, State: Running, Role: LEADER
I20260621 08:00:43.083871 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:43.083897 791 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8dd9e34a6d94a739b7482783eb4d794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260621 08:00:43.084511 786 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:43.085081 791 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8dd9e34a6d94a739b7482783eb4d794. Latest consensus state: current_term: 1 leader_uuid: "e8dd9e34a6d94a739b7482783eb4d794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8dd9e34a6d94a739b7482783eb4d794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } } }
I20260621 08:00:43.085259 791 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:43.085525 793 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8dd9e34a6d94a739b7482783eb4d794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8dd9e34a6d94a739b7482783eb4d794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } } }
I20260621 08:00:43.085635 793 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:43.085783 801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:43.089104 801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:43.091655 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:43.086423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:33797 (local address 127.0.0.1:42299)
0621 08:00:43.086517 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:43.086521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:43.086939 (+ 418us) server_negotiation.cc:407] Connection header received
0621 08:00:43.087106 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:43.087109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:43.087159 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:43.087228 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:43.087602 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:43.088357 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:43.089538 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:43.089843 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:43.089914 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:43.091136 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:43.091140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:43.091142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:43.091157 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:43.091184 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:43.091212 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:43.091271 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:43.091473 (+ 202us) server_negotiation.cc:299] Negotiation successful
0621 08:00:43.091506 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260621 08:00:43.100167 801 catalog_manager.cc:1347] Generated new cluster ID: 310cc3feedd34cabae3ecb8a4e178e8c
I20260621 08:00:43.100236 801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:43.120604 801 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:43.121577 801 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:43.129633 801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8dd9e34a6d94a739b7482783eb4d794: Generated new TSK 0
I20260621 08:00:43.130348 801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:43.137540 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 718
W20260621 08:00:43.142194 703 connection.cc:535] client connection to 127.0.0.1:42299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42299): Connection reset by peer (error 104)
2026-06-21T08:00:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:43 dist-test-slave-0pn3 krb5kdc[817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:43 dist-test-slave-0pn3 krb5kdc[817](info): set up 2 sockets
Jun 21 08:00:43 dist-test-slave-0pn3 krb5kdc[817](info): commencing operation
krb5kdc: starting...
W20260621 08:00:44.164774 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:44 dist-test-slave-0pn3 krb5kdc[817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:44.281453 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:44.390388 830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:44.390648 830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:44.390689 830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:44.394620 830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:44.394681 830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:44.394701 830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:44.394716 830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:44.394731 830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:44.399892 830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:44.400908 830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:44.401912 830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:44.410327 838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:44.410349 836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:44.410468 830 server_base.cc:1034] running on GCE node
W20260621 08:00:44.410678 835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:44.411258 830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:44.412351 830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:44.413542 830 hybrid_clock.cc:648] HybridClock initialized: now 1782028844413530 us; error 24 us; skew 500 ppm
Jun 21 08:00:44 dist-test-slave-0pn3 krb5kdc[817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:44.416158 830 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:44.417162 830 webserver.cc:460] Webserver started at http://127.0.0.1:42487/ using document root <none> and password file <none>
I20260621 08:00:44.417656 830 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:44.417711 830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:44.417877 830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:44.419837 830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "b94debfb28c44a66bb9ba590db56bc9c"
format_stamp: "Formatted at 2026-06-21 08:00:44 on dist-test-slave-0pn3"
server_key: "018deb155e97c3fd82b4dc6e86c5ba4a"
server_key_iv: "dadfbbfaf7fff333151e1d08c6943e6a"
server_key_version: "encryptionkey@0"
I20260621 08:00:44.420428 830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "b94debfb28c44a66bb9ba590db56bc9c"
format_stamp: "Formatted at 2026-06-21 08:00:44 on dist-test-slave-0pn3"
server_key: "018deb155e97c3fd82b4dc6e86c5ba4a"
server_key_iv: "dadfbbfaf7fff333151e1d08c6943e6a"
server_key_version: "encryptionkey@0"
I20260621 08:00:44.423897 830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:44.426234 845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:44.427546 830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:44.427667 830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "b94debfb28c44a66bb9ba590db56bc9c"
format_stamp: "Formatted at 2026-06-21 08:00:44 on dist-test-slave-0pn3"
server_key: "018deb155e97c3fd82b4dc6e86c5ba4a"
server_key_iv: "dadfbbfaf7fff333151e1d08c6943e6a"
server_key_version: "encryptionkey@0"
I20260621 08:00:44.427767 830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:44.432245 830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:44.435061 830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:44.435250 830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:44.442386 830 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36609
I20260621 08:00:44.442414 897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36609 every 8 connection(s)
I20260621 08:00:44.443288 830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:44.446056 898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:44.447705 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 830
I20260621 08:00:44.447849 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:44.448105 12158 external_mini_cluster.cc:1428] Setting key 2ba7c13f74bde9d7a89ef644acef9060
I20260621 08:00:44.452883 898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c: Bootstrap starting.
I20260621 08:00:44.453722 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:44.449557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:41843 (local address 127.0.0.1:36609)
0621 08:00:44.449964 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:44.449983 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:44.450024 (+ 41us) server_negotiation.cc:407] Connection header received
0621 08:00:44.450893 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:44.450918 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:44.451288 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:44.451657 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:44.452407 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:44.452412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:44.452428 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:44.452440 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:44.452500 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:44.452525 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:44.452656 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:44.452721 (+ 65us) server_negotiation.cc:299] Negotiation successful
0621 08:00:44.452869 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":103,"threads_started":1}
I20260621 08:00:44.455425 898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:44.456331 898 log.cc:824] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:44.456717 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:44.458583 898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c: No bootstrap required, opened a new log
I20260621 08:00:44.460563 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:44.458961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:48657 (local address 127.0.0.1:36609)
0621 08:00:44.459083 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:44.459087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:44.459246 (+ 159us) server_negotiation.cc:407] Connection header received
0621 08:00:44.459377 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:44.459380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:44.459406 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:44.459468 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:44.460158 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:44.460160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:44.460162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:44.460172 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:44.460194 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:44.460203 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:44.460266 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:44.460390 (+ 124us) server_negotiation.cc:299] Negotiation successful
0621 08:00:44.460421 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260621 08:00:44.462787 898 raft_consensus.cc:348] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94debfb28c44a66bb9ba590db56bc9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36609 } }
I20260621 08:00:44.462991 898 raft_consensus.cc:374] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:44.463061 898 raft_consensus.cc:729] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b94debfb28c44a66bb9ba590db56bc9c, State: Initialized, Role: FOLLOWER
I20260621 08:00:44.463455 898 consensus_queue.cc:260] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94debfb28c44a66bb9ba590db56bc9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36609 } }
I20260621 08:00:44.463591 898 raft_consensus.cc:388] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:44.463657 898 raft_consensus.cc:482] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:44.463766 898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:44.464819 898 raft_consensus.cc:504] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94debfb28c44a66bb9ba590db56bc9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36609 } }
I20260621 08:00:44.465111 898 leader_election.cc:302] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b94debfb28c44a66bb9ba590db56bc9c; no voters:
I20260621 08:00:44.465348 898 leader_election.cc:288] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:44.465464 908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:44.465690 908 raft_consensus.cc:686] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [term 1 LEADER]: Becoming Leader. State: Replica: b94debfb28c44a66bb9ba590db56bc9c, State: Running, Role: LEADER
I20260621 08:00:44.466012 908 consensus_queue.cc:237] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94debfb28c44a66bb9ba590db56bc9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36609 } }
I20260621 08:00:44.466571 898 sys_catalog.cc:564] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:44.467762 910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b94debfb28c44a66bb9ba590db56bc9c. Latest consensus state: current_term: 1 leader_uuid: "b94debfb28c44a66bb9ba590db56bc9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94debfb28c44a66bb9ba590db56bc9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36609 } } }
I20260621 08:00:44.467775 909 sys_catalog.cc:455] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b94debfb28c44a66bb9ba590db56bc9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94debfb28c44a66bb9ba590db56bc9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36609 } } }
I20260621 08:00:44.467959 910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:44.467988 909 sys_catalog.cc:458] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:44.468840 917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:44.471712 917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:44.477417 917 catalog_manager.cc:1347] Generated new cluster ID: 1e7db2bba1f144569ec5aaf190a6e6ee
I20260621 08:00:44.477489 917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:44.503262 917 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:44.503619 917 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:44.514055 917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b94debfb28c44a66bb9ba590db56bc9c: Generated new TSK 0
I20260621 08:00:44.514760 917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:44.517719 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 830
2026-06-21T08:00:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:44 dist-test-slave-0pn3 krb5kdc[929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:44 dist-test-slave-0pn3 krb5kdc[929](info): set up 2 sockets
Jun 21 08:00:44 dist-test-slave-0pn3 krb5kdc[929](info): commencing operation
krb5kdc: starting...
W20260621 08:00:45.552937 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:45 dist-test-slave-0pn3 krb5kdc[929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:45.667335 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:45.776017 942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:45.776258 942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:45.776298 942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:45.780066 942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:45.780113 942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:45.780131 942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:45.780145 942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:45.780158 942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:45.785012 942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:45.785984 942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:45.786969 942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:45.795266 950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:45.795225 948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:45.795236 947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:45.795367 942 server_base.cc:1034] running on GCE node
I20260621 08:00:45.796041 942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:45.797176 942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:45.798342 942 hybrid_clock.cc:648] HybridClock initialized: now 1782028845798309 us; error 40 us; skew 500 ppm
Jun 21 08:00:45 dist-test-slave-0pn3 krb5kdc[929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:45.801054 942 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:45.801954 942 webserver.cc:460] Webserver started at http://127.0.0.1:39027/ using document root <none> and password file <none>
I20260621 08:00:45.802520 942 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:45.802610 942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:45.802865 942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:45.805012 942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "08a0cb5eca554743ab59120b90b82311"
format_stamp: "Formatted at 2026-06-21 08:00:45 on dist-test-slave-0pn3"
server_key: "6b311cb5d9c25b6d3f8a40a5c7ccb9cd"
server_key_iv: "0586eefdff36d449b920acc647cc926e"
server_key_version: "encryptionkey@0"
I20260621 08:00:45.805576 942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "08a0cb5eca554743ab59120b90b82311"
format_stamp: "Formatted at 2026-06-21 08:00:45 on dist-test-slave-0pn3"
server_key: "6b311cb5d9c25b6d3f8a40a5c7ccb9cd"
server_key_iv: "0586eefdff36d449b920acc647cc926e"
server_key_version: "encryptionkey@0"
I20260621 08:00:45.809093 942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260621 08:00:45.811407 957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:45.812621 942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260621 08:00:45.812745 942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "08a0cb5eca554743ab59120b90b82311"
format_stamp: "Formatted at 2026-06-21 08:00:45 on dist-test-slave-0pn3"
server_key: "6b311cb5d9c25b6d3f8a40a5c7ccb9cd"
server_key_iv: "0586eefdff36d449b920acc647cc926e"
server_key_version: "encryptionkey@0"
I20260621 08:00:45.812839 942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:45.833676 942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:45.836805 942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:45.836958 942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:45.844357 942 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33073
I20260621 08:00:45.844381 1009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33073 every 8 connection(s)
I20260621 08:00:45.845336 942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:45.848002 1010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:45.853371 1010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311: Bootstrap starting.
I20260621 08:00:45.853470 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 942
I20260621 08:00:45.853629 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:45.853892 12158 external_mini_cluster.cc:1428] Setting key 411b369ff3e8714715a06a8fede693e7
I20260621 08:00:45.856055 1010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:45.856932 1010 log.cc:824] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:45.859077 1010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311: No bootstrap required, opened a new log
I20260621 08:00:45.862989 1010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a0cb5eca554743ab59120b90b82311" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
Jun 21 08:00:45 dist-test-slave-0pn3 krb5kdc[929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260621 08:00:45.863233 1010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:45.863314 1010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a0cb5eca554743ab59120b90b82311, State: Initialized, Role: FOLLOWER
I20260621 08:00:45.863794 1010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a0cb5eca554743ab59120b90b82311" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260621 08:00:45.863965 1010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:45.864073 1010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:45.864183 1010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:45.865552 1010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a0cb5eca554743ab59120b90b82311" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260621 08:00:45.865935 1010 leader_election.cc:302] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08a0cb5eca554743ab59120b90b82311; no voters:
I20260621 08:00:45.866302 1010 leader_election.cc:288] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:45.866386 1015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:45.866588 1015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [term 1 LEADER]: Becoming Leader. State: Replica: 08a0cb5eca554743ab59120b90b82311, State: Running, Role: LEADER
I20260621 08:00:45.866925 1015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a0cb5eca554743ab59120b90b82311" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260621 08:00:45.867611 1010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:45.868839 1016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08a0cb5eca554743ab59120b90b82311" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a0cb5eca554743ab59120b90b82311" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } } }
I20260621 08:00:45.868975 1016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:45.869375 1024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:45.869329 1017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08a0cb5eca554743ab59120b90b82311. Latest consensus state: current_term: 1 leader_uuid: "08a0cb5eca554743ab59120b90b82311" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a0cb5eca554743ab59120b90b82311" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } } }
I20260621 08:00:45.869481 1017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:45.869490 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:45.855222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:52473 (local address 127.0.0.1:33073)
0621 08:00:45.855672 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:45.855690 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:45.855730 (+ 40us) server_negotiation.cc:407] Connection header received
0621 08:00:45.856612 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:45.856635 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:45.857071 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:45.857440 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:45.858211 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:45.859599 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:45.860447 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:45.861130 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:45.861303 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:45.863874 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:45.863879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:45.863895 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:45.863928 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:45.866003 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:45.866638 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:45.866644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:45.866651 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:45.866779 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:45.867150 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:45.867152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:45.867154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:45.867549 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:45.867714 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:45.868242 (+ 528us) server_negotiation.cc:299] Negotiation successful
0621 08:00:45.868514 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":120,"threads_started":1}
I20260621 08:00:45.872655 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:45.872915 1024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:45.878298 1024 catalog_manager.cc:1347] Generated new cluster ID: 450c19c006b746ebbb2ecd1f12d7aa5f
I20260621 08:00:45.878367 1024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:45.886034 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:45.876839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:47749 (local address 127.0.0.1:33073)
0621 08:00:45.876966 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:45.876970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:45.878589 (+ 1619us) server_negotiation.cc:407] Connection header received
0621 08:00:45.878742 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:45.878745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:45.878794 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:45.878878 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:45.879534 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:45.880079 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:45.880947 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:45.881188 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:45.881294 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:45.882729 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:45.882733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:45.882735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:45.882761 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:45.884227 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:45.884744 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:45.884747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:45.884749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:45.884826 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:45.885155 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:45.885158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:45.885160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:45.885328 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:45.885476 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:45.885756 (+ 280us) server_negotiation.cc:299] Negotiation successful
0621 08:00:45.885861 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 08:00:45.898721 1024 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:45.899619 1024 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:45.905613 1024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08a0cb5eca554743ab59120b90b82311: Generated new TSK 0
I20260621 08:00:45.906358 1024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:45.911664 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 942
W20260621 08:00:45.916970 927 connection.cc:535] client connection to 127.0.0.1:33073 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33073): Connection reset by peer (error 104)
2026-06-21T08:00:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:45 dist-test-slave-0pn3 krb5kdc[1041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:45 dist-test-slave-0pn3 krb5kdc[1041](info): set up 2 sockets
Jun 21 08:00:45 dist-test-slave-0pn3 krb5kdc[1041](info): commencing operation
krb5kdc: starting...
W20260621 08:00:46.944890 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:47 dist-test-slave-0pn3 krb5kdc[1041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:47.064630 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:47.178500 1054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:47.178838 1054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:47.178915 1054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:47.183020 1054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:47.183123 1054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:47.183162 1054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:47.183220 1054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:47.183260 1054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:47.188701 1054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:47.189819 1054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:47.190889 1054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260621 08:00:47.199818 1054 server_base.cc:1034] running on GCE node
W20260621 08:00:47.199726 1062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:47.199740 1060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:47.199762 1059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:47.200685 1054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:47.201807 1054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:47.203012 1054 hybrid_clock.cc:648] HybridClock initialized: now 1782028847202994 us; error 34 us; skew 500 ppm
Jun 21 08:00:47 dist-test-slave-0pn3 krb5kdc[1041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:47.206746 1054 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:47.207953 1054 webserver.cc:460] Webserver started at http://127.0.0.1:43699/ using document root <none> and password file <none>
I20260621 08:00:47.208721 1054 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:47.208837 1054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:47.209157 1054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:47.211891 1054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca"
format_stamp: "Formatted at 2026-06-21 08:00:47 on dist-test-slave-0pn3"
server_key: "b99b8df87f539c80508d07af035c852a"
server_key_iv: "8c38856c07b8b0ed5113bcfc14f741d6"
server_key_version: "encryptionkey@0"
I20260621 08:00:47.212680 1054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca"
format_stamp: "Formatted at 2026-06-21 08:00:47 on dist-test-slave-0pn3"
server_key: "b99b8df87f539c80508d07af035c852a"
server_key_iv: "8c38856c07b8b0ed5113bcfc14f741d6"
server_key_version: "encryptionkey@0"
I20260621 08:00:47.216450 1054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260621 08:00:47.218731 1069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:47.219942 1054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:47.220072 1054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca"
format_stamp: "Formatted at 2026-06-21 08:00:47 on dist-test-slave-0pn3"
server_key: "b99b8df87f539c80508d07af035c852a"
server_key_iv: "8c38856c07b8b0ed5113bcfc14f741d6"
server_key_version: "encryptionkey@0"
I20260621 08:00:47.220183 1054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:47.244793 1054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:47.247766 1054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:47.247975 1054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:47.255430 1054 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39307
I20260621 08:00:47.255453 1121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39307 every 8 connection(s)
I20260621 08:00:47.256434 1054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:47.259181 1122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:47.261384 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1054
I20260621 08:00:47.261533 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:47.261843 12158 external_mini_cluster.cc:1428] Setting key 93b1a7d25579b6aa7aa72d852976af00
I20260621 08:00:47.265497 1122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca: Bootstrap starting.
I20260621 08:00:47.267760 1122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:47.268538 1122 log.cc:824] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:47.270900 1122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca: No bootstrap required, opened a new log
I20260621 08:00:47.271448 1125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:47.263235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:36787 (local address 127.0.0.1:39307)
0621 08:00:47.263662 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:47.263677 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:47.263725 (+ 48us) server_negotiation.cc:407] Connection header received
0621 08:00:47.264610 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:47.264629 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:47.265023 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:47.265372 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:47.265775 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:47.266759 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:47.267735 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:47.268256 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:47.268541 (+ 285us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:47.269503 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:47.269511 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:47.269530 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:47.269992 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:47.270058 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:47.270089 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:47.270229 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:47.270404 (+ 175us) server_negotiation.cc:299] Negotiation successful
0621 08:00:47.270533 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
I20260621 08:00:47.274505 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:47.274765 1122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260621 08:00:47.275005 1122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:47.275079 1122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 479bba51ccd74fcbb2ddb3e497c7f4ca, State: Initialized, Role: FOLLOWER
I20260621 08:00:47.275502 1122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260621 08:00:47.275615 1122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:47.275695 1122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:47.275821 1122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:47.277070 1122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260621 08:00:47.277413 1122 leader_election.cc:302] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 479bba51ccd74fcbb2ddb3e497c7f4ca; no voters:
I20260621 08:00:47.277704 1122 leader_election.cc:288] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:47.277849 1132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:47.278096 1132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [term 1 LEADER]: Becoming Leader. State: Replica: 479bba51ccd74fcbb2ddb3e497c7f4ca, State: Running, Role: LEADER
I20260621 08:00:47.278404 1132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } }
I20260621 08:00:47.278927 1122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:47.280153 1134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 479bba51ccd74fcbb2ddb3e497c7f4ca. Latest consensus state: current_term: 1 leader_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } } }
I20260621 08:00:47.280150 1133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479bba51ccd74fcbb2ddb3e497c7f4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39307 } } }
I20260621 08:00:47.280365 1133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:47.280638 1134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:47.284252 1147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:47.284359 1147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:47.284435 1144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:47.285333 1144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:47.286057 1125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:47.276741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:50859 (local address 127.0.0.1:39307)
0621 08:00:47.276856 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:47.276860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:47.281639 (+ 4779us) server_negotiation.cc:407] Connection header received
0621 08:00:47.281730 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:47.281733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:47.281782 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:47.281861 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:47.282101 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:47.282836 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:47.283937 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:47.284241 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:47.284363 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:47.285519 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:47.285523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:47.285526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:47.285540 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:47.285570 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:47.285584 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:47.285674 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:47.285882 (+ 208us) server_negotiation.cc:299] Negotiation successful
0621 08:00:47.285921 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 08:00:47.290616 1144 catalog_manager.cc:1347] Generated new cluster ID: 4e276a3bc2da4eeba9b159e6dd9dd270
I20260621 08:00:47.290684 1144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:47.306901 1144 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:47.308156 1144 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:47.313484 1144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 479bba51ccd74fcbb2ddb3e497c7f4ca: Generated new TSK 0
I20260621 08:00:47.314285 1144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:47.326049 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1054
W20260621 08:00:47.331352 1039 connection.cc:535] client connection to 127.0.0.1:39307 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39307): Connection reset by peer (error 104)
2026-06-21T08:00:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:47 dist-test-slave-0pn3 krb5kdc[1153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:47 dist-test-slave-0pn3 krb5kdc[1153](info): set up 2 sockets
Jun 21 08:00:47 dist-test-slave-0pn3 krb5kdc[1153](info): commencing operation
krb5kdc: starting...
W20260621 08:00:48.344784 12158 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:48 dist-test-slave-0pn3 krb5kdc[1153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260621 08:00:48.461104 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:48.572495 1166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:48.572762 1166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:48.572805 1166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:48.576805 1166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:48.576862 1166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:48.576881 1166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:48.576898 1166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:48.576957 1166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:48.582172 1166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:48.583283 1166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:48.584390 1166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:48.592734 1172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:48.592906 1171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:48.592876 1166 server_base.cc:1034] running on GCE node
W20260621 08:00:48.592896 1174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:48.593708 1166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:48.594787 1166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:48.595949 1166 hybrid_clock.cc:648] HybridClock initialized: now 1782028848595931 us; error 31 us; skew 500 ppm
Jun 21 08:00:48 dist-test-slave-0pn3 krb5kdc[1153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:48.601766 1166 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260621 08:00:48.602751 1166 webserver.cc:460] Webserver started at http://127.0.0.1:35981/ using document root <none> and password file <none>
I20260621 08:00:48.603346 1166 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:48.603407 1166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:48.603650 1166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:48.605815 1166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "9b7e989d9bd747f3925248a0eaabaaa1"
format_stamp: "Formatted at 2026-06-21 08:00:48 on dist-test-slave-0pn3"
server_key: "76b602e2a36e326b521503521443cd46"
server_key_iv: "78ba1f74ff2f0eb0ffa5a854d07a899f"
server_key_version: "encryptionkey@0"
I20260621 08:00:48.606379 1166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "9b7e989d9bd747f3925248a0eaabaaa1"
format_stamp: "Formatted at 2026-06-21 08:00:48 on dist-test-slave-0pn3"
server_key: "76b602e2a36e326b521503521443cd46"
server_key_iv: "78ba1f74ff2f0eb0ffa5a854d07a899f"
server_key_version: "encryptionkey@0"
I20260621 08:00:48.610174 1166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260621 08:00:48.612582 1181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:48.613817 1166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 08:00:48.613958 1166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "9b7e989d9bd747f3925248a0eaabaaa1"
format_stamp: "Formatted at 2026-06-21 08:00:48 on dist-test-slave-0pn3"
server_key: "76b602e2a36e326b521503521443cd46"
server_key_iv: "78ba1f74ff2f0eb0ffa5a854d07a899f"
server_key_version: "encryptionkey@0"
I20260621 08:00:48.614069 1166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:48.618693 1166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:48.621603 1166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:48.621798 1166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:48.628615 1166 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39749
I20260621 08:00:48.628641 1233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39749 every 8 connection(s)
I20260621 08:00:48.629537 1166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:48.632423 1234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:48.637042 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1166
I20260621 08:00:48.637221 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:48.637514 12158 external_mini_cluster.cc:1428] Setting key 5c9c28c889441841783f29783e69e76c
I20260621 08:00:48.638875 1234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1: Bootstrap starting.
I20260621 08:00:48.641376 1234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:48.642113 1234 log.cc:824] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:48.642999 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:48.639024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:47315 (local address 127.0.0.1:39749)
0621 08:00:48.639470 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:48.639485 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:48.639521 (+ 36us) server_negotiation.cc:407] Connection header received
0621 08:00:48.640181 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:48.640199 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:48.640572 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:48.640915 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:48.641785 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:48.641790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:48.641802 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:48.641813 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:48.641865 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:48.641884 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:48.642002 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:48.642105 (+ 103us) server_negotiation.cc:299] Negotiation successful
0621 08:00:48.642292 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":126,"threads_started":1}
I20260621 08:00:48.644397 1234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1: No bootstrap required, opened a new log
I20260621 08:00:48.645846 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:48.648515 1234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39749 } }
I20260621 08:00:48.648764 1234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:48.648840 1234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b7e989d9bd747f3925248a0eaabaaa1, State: Initialized, Role: FOLLOWER
I20260621 08:00:48.649303 1234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39749 } }
I20260621 08:00:48.649475 1234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:48.649571 1234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:48.649678 1234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:48.650130 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:48.647856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:48319 (local address 127.0.0.1:39749)
0621 08:00:48.647963 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:48.647967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:48.648577 (+ 610us) server_negotiation.cc:407] Connection header received
0621 08:00:48.648720 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:48.648723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:48.648758 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:48.648847 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:48.649719 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:48.649722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:48.649725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:48.649736 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:48.649761 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:48.649784 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:48.649861 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:48.649986 (+ 125us) server_negotiation.cc:299] Negotiation successful
0621 08:00:48.650017 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 08:00:48.650897 1234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39749 } }
I20260621 08:00:48.651234 1234 leader_election.cc:302] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b7e989d9bd747f3925248a0eaabaaa1; no voters:
I20260621 08:00:48.651551 1234 leader_election.cc:288] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:48.651621 1244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:48.651872 1244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [term 1 LEADER]: Becoming Leader. State: Replica: 9b7e989d9bd747f3925248a0eaabaaa1, State: Running, Role: LEADER
I20260621 08:00:48.652210 1244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39749 } }
I20260621 08:00:48.652825 1234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:48.653828 1246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b7e989d9bd747f3925248a0eaabaaa1. Latest consensus state: current_term: 1 leader_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39749 } } }
I20260621 08:00:48.654181 1246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:48.654088 1245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e989d9bd747f3925248a0eaabaaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39749 } } }
I20260621 08:00:48.654536 1245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:48.654615 1253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:48.657506 1253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:48.662730 1253 catalog_manager.cc:1347] Generated new cluster ID: f0e9616ca24f4b0a8452d10bee6bc25f
I20260621 08:00:48.662804 1253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:48.680857 1253 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:48.681223 1253 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:48.688966 1253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b7e989d9bd747f3925248a0eaabaaa1: Generated new TSK 0
I20260621 08:00:48.689692 1253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:48.698530 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1166
2026-06-21T08:00:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:48 dist-test-slave-0pn3 krb5kdc[1265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:48 dist-test-slave-0pn3 krb5kdc[1265](info): set up 2 sockets
Jun 21 08:00:48 dist-test-slave-0pn3 krb5kdc[1265](info): commencing operation
krb5kdc: starting...
W20260621 08:00:49.732865 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:49 dist-test-slave-0pn3 krb5kdc[1265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:49.853336 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:34571
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:34571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:49.965225 1278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:49.965615 1278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:49.965703 1278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:49.973420 1278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:49.973516 1278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:49.973552 1278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:49.973588 1278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:49.973630 1278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:49.979169 1278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:34571
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:34571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:49.980402 1278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:49.981484 1278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:49.990227 1284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:49.990213 1283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:49.990260 1278 server_base.cc:1034] running on GCE node
W20260621 08:00:49.990219 1286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:49.990988 1278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:49.992105 1278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:49.993281 1278 hybrid_clock.cc:648] HybridClock initialized: now 1782028849993267 us; error 58 us; skew 500 ppm
Jun 21 08:00:49 dist-test-slave-0pn3 krb5kdc[1265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028849, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:49.995927 1278 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:49.996934 1278 webserver.cc:460] Webserver started at http://127.11.223.190:46073/ using document root <none> and password file <none>
I20260621 08:00:49.997515 1278 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:49.997608 1278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:49.997843 1278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:49.999890 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "3be428ec278c44e3a9fca8ed405b10ec"
format_stamp: "Formatted at 2026-06-21 08:00:49 on dist-test-slave-0pn3"
server_key: "0273db05d4bdfea6fe50b898f4ce40a5"
server_key_iv: "e58fe9fbe2265f9cbedf8345e46b9b38"
server_key_version: "encryptionkey@0"
I20260621 08:00:50.000514 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "3be428ec278c44e3a9fca8ed405b10ec"
format_stamp: "Formatted at 2026-06-21 08:00:49 on dist-test-slave-0pn3"
server_key: "0273db05d4bdfea6fe50b898f4ce40a5"
server_key_iv: "e58fe9fbe2265f9cbedf8345e46b9b38"
server_key_version: "encryptionkey@0"
I20260621 08:00:50.004267 1278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:50.006651 1293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:50.007818 1278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:50.007964 1278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "3be428ec278c44e3a9fca8ed405b10ec"
format_stamp: "Formatted at 2026-06-21 08:00:49 on dist-test-slave-0pn3"
server_key: "0273db05d4bdfea6fe50b898f4ce40a5"
server_key_iv: "e58fe9fbe2265f9cbedf8345e46b9b38"
server_key_version: "encryptionkey@0"
I20260621 08:00:50.008077 1278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:50.027554 1278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:50.030975 1278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:50.031206 1278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:50.039417 1278 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:34571
I20260621 08:00:50.039728 1345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:34571 every 8 connection(s)
I20260621 08:00:50.040777 1278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:50.044665 1346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:50.049760 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1278
I20260621 08:00:50.049966 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:50.050225 12158 external_mini_cluster.cc:1428] Setting key 2859f12ffe97d48cd47a92b2dee46a8f
I20260621 08:00:50.052780 1346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec: Bootstrap starting.
I20260621 08:00:50.055872 1346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:50.056891 1346 log.cc:824] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:50.059322 1346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec: No bootstrap required, opened a new log
Jun 21 08:00:50 dist-test-slave-0pn3 krb5kdc[1265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:50.063570 1346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be428ec278c44e3a9fca8ed405b10ec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34571 } }
I20260621 08:00:50.063839 1346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:50.063963 1346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be428ec278c44e3a9fca8ed405b10ec, State: Initialized, Role: FOLLOWER
I20260621 08:00:50.064528 1346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be428ec278c44e3a9fca8ed405b10ec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34571 } }
I20260621 08:00:50.064718 1346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:50.064828 1346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:50.064929 1346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:50.066118 1346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be428ec278c44e3a9fca8ed405b10ec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34571 } }
I20260621 08:00:50.066481 1346 leader_election.cc:302] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3be428ec278c44e3a9fca8ed405b10ec; no voters:
I20260621 08:00:50.066788 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:50.051671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55227 (local address 127.11.223.190:34571)
0621 08:00:50.052098 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:50.052113 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:50.052148 (+ 35us) server_negotiation.cc:407] Connection header received
0621 08:00:50.052833 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:50.052850 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:50.053238 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:50.053595 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:50.054431 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:50.055430 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:50.056459 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:50.056929 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:50.057186 (+ 257us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:50.060449 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:50.060455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:50.060470 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:50.060501 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:50.063184 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:50.063838 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:50.063844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:50.063853 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:50.064001 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:50.064399 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:50.064402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:50.064404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:50.064965 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:50.065182 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:50.065543 (+ 361us) server_negotiation.cc:299] Negotiation successful
0621 08:00:50.065769 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260621 08:00:50.066823 1346 leader_election.cc:288] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:50.068341 1346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:50.066907 1351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:50.070344 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:50.070504 1351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [term 1 LEADER]: Becoming Leader. State: Replica: 3be428ec278c44e3a9fca8ed405b10ec, State: Running, Role: LEADER
I20260621 08:00:50.070887 1351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be428ec278c44e3a9fca8ed405b10ec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34571 } }
I20260621 08:00:50.073446 1363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3be428ec278c44e3a9fca8ed405b10ec. Latest consensus state: current_term: 1 leader_uuid: "3be428ec278c44e3a9fca8ed405b10ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be428ec278c44e3a9fca8ed405b10ec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34571 } } }
I20260621 08:00:50.073575 1363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:50.074128 1352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3be428ec278c44e3a9fca8ed405b10ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be428ec278c44e3a9fca8ed405b10ec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34571 } } }
W20260621 08:00:50.076380 1370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:50.076483 1370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:50.075907 1352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:50.076952 1371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:50.077970 1371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:50.083601 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:50.072572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51035 (local address 127.11.223.190:34571)
0621 08:00:50.072689 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:50.072693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:50.073104 (+ 411us) server_negotiation.cc:407] Connection header received
0621 08:00:50.073797 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:50.073806 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:50.073862 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:50.073944 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:50.075020 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:50.075785 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:50.077026 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:50.077353 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:50.077450 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:50.079374 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:50.079380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:50.079385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:50.079411 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:50.081186 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:50.081786 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:50.081790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:50.081792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:50.081876 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:50.082262 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:50.082266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:50.082268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:50.082478 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:50.082626 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:50.083290 (+ 664us) server_negotiation.cc:299] Negotiation successful
0621 08:00:50.083398 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 08:00:50.085511 1371 catalog_manager.cc:1347] Generated new cluster ID: d75ebdf8c9b644ab99641f51d4207865
I20260621 08:00:50.085613 1371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:50.112033 1371 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:50.113273 1371 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:50.123826 1371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3be428ec278c44e3a9fca8ed405b10ec: Generated new TSK 0
I20260621 08:00:50.124817 1371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:50.132160 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1278
W20260621 08:00:50.137584 1263 connection.cc:535] client connection to 127.11.223.190:34571 recv error: Network error: failed to read from TLS socket (remote: 127.11.223.190:34571): Connection reset by peer (error 104)
2026-06-21T08:00:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:50 dist-test-slave-0pn3 krb5kdc[1377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:50 dist-test-slave-0pn3 krb5kdc[1377](info): set up 2 sockets
Jun 21 08:00:50 dist-test-slave-0pn3 krb5kdc[1377](info): commencing operation
krb5kdc: starting...
W20260621 08:00:51.164839 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:51 dist-test-slave-0pn3 krb5kdc[1377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:51.282471 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45975
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:51.388499 1390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:51.388804 1390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:51.388875 1390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:51.392724 1390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:51.392849 1390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:51.392887 1390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:51.392947 1390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:51.392987 1390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:51.398115 1390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:51.399137 1390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:51.400144 1390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:51.408571 1395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:51.408555 1396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:51.408557 1398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:51.408774 1390 server_base.cc:1034] running on GCE node
I20260621 08:00:51.409415 1390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:51.410435 1390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:51.411618 1390 hybrid_clock.cc:648] HybridClock initialized: now 1782028851411587 us; error 45 us; skew 500 ppm
Jun 21 08:00:51 dist-test-slave-0pn3 krb5kdc[1377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028851, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:51.414232 1390 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:51.415089 1390 webserver.cc:460] Webserver started at http://127.11.223.190:38429/ using document root <none> and password file <none>
I20260621 08:00:51.415647 1390 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:51.415737 1390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:51.415961 1390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:51.418097 1390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "5c2ea0bda3e748248f2cdbddd8be1701"
format_stamp: "Formatted at 2026-06-21 08:00:51 on dist-test-slave-0pn3"
server_key: "ef1664a8ebfdd8380316ce227885ca72"
server_key_iv: "1a04c8a18211cd23048b8df7ba60ccd6"
server_key_version: "encryptionkey@0"
I20260621 08:00:51.418790 1390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "5c2ea0bda3e748248f2cdbddd8be1701"
format_stamp: "Formatted at 2026-06-21 08:00:51 on dist-test-slave-0pn3"
server_key: "ef1664a8ebfdd8380316ce227885ca72"
server_key_iv: "1a04c8a18211cd23048b8df7ba60ccd6"
server_key_version: "encryptionkey@0"
I20260621 08:00:51.422447 1390 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260621 08:00:51.424899 1405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:51.426378 1390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260621 08:00:51.426493 1390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "5c2ea0bda3e748248f2cdbddd8be1701"
format_stamp: "Formatted at 2026-06-21 08:00:51 on dist-test-slave-0pn3"
server_key: "ef1664a8ebfdd8380316ce227885ca72"
server_key_iv: "1a04c8a18211cd23048b8df7ba60ccd6"
server_key_version: "encryptionkey@0"
I20260621 08:00:51.426602 1390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:51.439023 1390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:51.441825 1390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:51.441972 1390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:51.448908 1390 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:45975
I20260621 08:00:51.448916 1457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:45975 every 8 connection(s)
I20260621 08:00:51.449860 1390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:51.452612 1458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:51.457870 1458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701: Bootstrap starting.
I20260621 08:00:51.458154 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1390
I20260621 08:00:51.458300 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:51.458570 12158 external_mini_cluster.cc:1428] Setting key c53c4e82c1d7f212293ce40852afe058
I20260621 08:00:51.460505 1458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:51.461198 1458 log.cc:824] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:51.462996 1458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701: No bootstrap required, opened a new log
I20260621 08:00:51.466429 1458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45975 } }
I20260621 08:00:51.466748 1458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:51.466830 1458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2ea0bda3e748248f2cdbddd8be1701, State: Initialized, Role: FOLLOWER
I20260621 08:00:51.467294 1458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45975 } }
I20260621 08:00:51.467404 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:51.459908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49927 (local address 127.11.223.190:45975)
0621 08:00:51.460304 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:51.460366 (+ 62us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:51.460409 (+ 43us) server_negotiation.cc:407] Connection header received
0621 08:00:51.461056 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:51.461073 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:51.461453 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:51.461760 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:51.462151 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:51.463187 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:51.464057 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:51.464596 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:51.464798 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:51.465691 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:51.465696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:51.465707 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:51.466072 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:51.466121 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:51.466142 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:51.466279 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:51.466460 (+ 181us) server_negotiation.cc:299] Negotiation successful
0621 08:00:51.466587 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":130,"threads_started":1}
I20260621 08:00:51.467437 1458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:51.467576 1458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:51.467675 1458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:51.468664 1458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45975 } }
I20260621 08:00:51.468981 1458 leader_election.cc:302] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c2ea0bda3e748248f2cdbddd8be1701; no voters:
I20260621 08:00:51.469347 1458 leader_election.cc:288] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:51.469434 1463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:51.469647 1463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [term 1 LEADER]: Becoming Leader. State: Replica: 5c2ea0bda3e748248f2cdbddd8be1701, State: Running, Role: LEADER
I20260621 08:00:51.469965 1463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45975 } }
I20260621 08:00:51.470357 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:51.470532 1458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:51.471117 1464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c2ea0bda3e748248f2cdbddd8be1701. Latest consensus state: current_term: 1 leader_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45975 } } }
I20260621 08:00:51.471242 1464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:51.471632 1465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2ea0bda3e748248f2cdbddd8be1701" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45975 } } }
I20260621 08:00:51.471776 1465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:51.472074 1470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:51.475255 1470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:51.477856 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:51.473058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40885 (local address 127.11.223.190:45975)
0621 08:00:51.473158 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:51.473162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:51.473534 (+ 372us) server_negotiation.cc:407] Connection header received
0621 08:00:51.473693 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:51.473697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:51.473738 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:51.473808 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:51.474153 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:51.474961 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:51.476061 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:51.476355 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:51.476449 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:51.477315 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:51.477319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:51.477321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:51.477335 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:51.477363 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:51.477376 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:51.477458 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:51.477682 (+ 224us) server_negotiation.cc:299] Negotiation successful
0621 08:00:51.477716 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260621 08:00:51.481017 1470 catalog_manager.cc:1347] Generated new cluster ID: 9610a63e1ebf4d60b405933a9fbd21a7
I20260621 08:00:51.481081 1470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:51.492873 1470 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:51.494071 1470 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:51.498412 1470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c2ea0bda3e748248f2cdbddd8be1701: Generated new TSK 0
I20260621 08:00:51.499046 1470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:51.505399 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1390
W20260621 08:00:51.510617 1375 connection.cc:535] client connection to 127.11.223.190:45975 recv error: Network error: failed to read from TLS socket (remote: 127.11.223.190:45975): Connection reset by peer (error 104)
2026-06-21T08:00:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:51 dist-test-slave-0pn3 krb5kdc[1489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:51 dist-test-slave-0pn3 krb5kdc[1489](info): set up 2 sockets
Jun 21 08:00:51 dist-test-slave-0pn3 krb5kdc[1489](info): commencing operation
krb5kdc: starting...
W20260621 08:00:52.525794 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:52 dist-test-slave-0pn3 krb5kdc[1489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:52.643392 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:35251
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:35251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:52.750293 1502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:52.750531 1502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:52.750571 1502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:52.754483 1502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:52.754537 1502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:52.754556 1502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:52.754575 1502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:52.754590 1502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:52.759675 1502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:35251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:35251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:52.760679 1502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:52.761607 1502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:52.769469 1507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:52.769479 1510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:52.769483 1508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:52.769896 1502 server_base.cc:1034] running on GCE node
I20260621 08:00:52.770413 1502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:52.771418 1502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:52.772569 1502 hybrid_clock.cc:648] HybridClock initialized: now 1782028852772543 us; error 40 us; skew 500 ppm
Jun 21 08:00:52 dist-test-slave-0pn3 krb5kdc[1489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:52.775102 1502 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:52.775900 1502 webserver.cc:460] Webserver started at http://127.11.223.190:36113/ using document root <none> and password file <none>
I20260621 08:00:52.776486 1502 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:52.776575 1502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:52.776808 1502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:52.778731 1502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "cb86490cbe8b4f85b45a8a3f929121b7"
format_stamp: "Formatted at 2026-06-21 08:00:52 on dist-test-slave-0pn3"
server_key: "f4f0bdb4b7b0238e261da89ab50438d7"
server_key_iv: "9833ebc2ee0d02900f52501926a3a192"
server_key_version: "encryptionkey@0"
I20260621 08:00:52.779278 1502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "cb86490cbe8b4f85b45a8a3f929121b7"
format_stamp: "Formatted at 2026-06-21 08:00:52 on dist-test-slave-0pn3"
server_key: "f4f0bdb4b7b0238e261da89ab50438d7"
server_key_iv: "9833ebc2ee0d02900f52501926a3a192"
server_key_version: "encryptionkey@0"
I20260621 08:00:52.782742 1502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260621 08:00:52.785022 1517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:52.786123 1502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:52.786264 1502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "cb86490cbe8b4f85b45a8a3f929121b7"
format_stamp: "Formatted at 2026-06-21 08:00:52 on dist-test-slave-0pn3"
server_key: "f4f0bdb4b7b0238e261da89ab50438d7"
server_key_iv: "9833ebc2ee0d02900f52501926a3a192"
server_key_version: "encryptionkey@0"
I20260621 08:00:52.786369 1502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:52.790642 1502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:52.793416 1502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:52.793601 1502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:52.800246 1502 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:35251
I20260621 08:00:52.800302 1569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:35251 every 8 connection(s)
I20260621 08:00:52.801208 1502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:52.803821 1570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:52.809069 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1502
I20260621 08:00:52.809201 1570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7: Bootstrap starting.
I20260621 08:00:52.809237 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:52.809583 12158 external_mini_cluster.cc:1428] Setting key deda979e9d9a09a40c3782b09f2e12fd
I20260621 08:00:52.811765 1570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:52.812551 1570 log.cc:824] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:52.814574 1570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7: No bootstrap required, opened a new log
I20260621 08:00:52.814620 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:52.810828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36037 (local address 127.11.223.190:35251)
0621 08:00:52.811289 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:52.811304 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:52.811337 (+ 33us) server_negotiation.cc:407] Connection header received
0621 08:00:52.811989 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:52.812005 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:52.812287 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:52.812656 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:52.813470 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:52.813474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:52.813486 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:52.813495 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:52.813547 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:52.813566 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:52.813681 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:52.813783 (+ 102us) server_negotiation.cc:299] Negotiation successful
0621 08:00:52.813939 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260621 08:00:52.817320 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:52.818596 1570 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35251 } }
I20260621 08:00:52.818827 1570 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:52.818903 1570 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb86490cbe8b4f85b45a8a3f929121b7, State: Initialized, Role: FOLLOWER
I20260621 08:00:52.819391 1570 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35251 } }
I20260621 08:00:52.819562 1570 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:52.819645 1570 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:52.819751 1570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:52.820858 1570 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35251 } }
I20260621 08:00:52.820993 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:52.819246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48803 (local address 127.11.223.190:35251)
0621 08:00:52.819377 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:52.819382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:52.819648 (+ 266us) server_negotiation.cc:407] Connection header received
0621 08:00:52.819789 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:52.819791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:52.819837 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:52.819917 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:52.820614 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:52.820617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:52.820620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:52.820630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:52.820653 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:52.820664 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:52.820757 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:52.820829 (+ 72us) server_negotiation.cc:299] Negotiation successful
0621 08:00:52.820866 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260621 08:00:52.821265 1570 leader_election.cc:302] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb86490cbe8b4f85b45a8a3f929121b7; no voters:
I20260621 08:00:52.821636 1570 leader_election.cc:288] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:52.821703 1580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:52.822000 1580 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [term 1 LEADER]: Becoming Leader. State: Replica: cb86490cbe8b4f85b45a8a3f929121b7, State: Running, Role: LEADER
I20260621 08:00:52.822270 1580 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35251 } }
I20260621 08:00:52.822875 1570 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:52.823421 1580 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb86490cbe8b4f85b45a8a3f929121b7. Latest consensus state: current_term: 1 leader_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35251 } } }
I20260621 08:00:52.823612 1580 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:52.823871 1581 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb86490cbe8b4f85b45a8a3f929121b7" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35251 } } }
I20260621 08:00:52.824007 1581 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:52.824054 1588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:52.827174 1588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:52.832571 1588 catalog_manager.cc:1347] Generated new cluster ID: 67f21c85341c448db11ad0dbcfd2d146
I20260621 08:00:52.832647 1588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:52.851681 1588 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:52.852001 1588 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:52.858362 1588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb86490cbe8b4f85b45a8a3f929121b7: Generated new TSK 0
I20260621 08:00:52.859057 1588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:52.871065 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1502
2026-06-21T08:00:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:52 dist-test-slave-0pn3 krb5kdc[1601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:52 dist-test-slave-0pn3 krb5kdc[1601](info): set up 2 sockets
Jun 21 08:00:52 dist-test-slave-0pn3 krb5kdc[1601](info): commencing operation
krb5kdc: starting...
W20260621 08:00:53.904892 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:53 dist-test-slave-0pn3 krb5kdc[1601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:54.021620 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44519
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:44519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:54.134501 1614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:54.134773 1614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:54.134817 1614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:54.139034 1614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:54.139103 1614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:54.139123 1614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:54.139140 1614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:54.139156 1614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:54.144649 1614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:44519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:44519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:54.145738 1614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:54.146790 1614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:54.155378 1622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:54.155373 1620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:54.155715 1614 server_base.cc:1034] running on GCE node
W20260621 08:00:54.155373 1619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:54.156539 1614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:54.157756 1614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:54.158963 1614 hybrid_clock.cc:648] HybridClock initialized: now 1782028854158948 us; error 39 us; skew 500 ppm
Jun 21 08:00:54 dist-test-slave-0pn3 krb5kdc[1601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028854, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:54.165548 1614 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:54.166515 1614 webserver.cc:460] Webserver started at http://127.11.223.190:34917/ using document root <none> and password file <none>
I20260621 08:00:54.167058 1614 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:54.167128 1614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:54.167297 1614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:54.169442 1614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "53bc78cecc9b4da6a21e2e35d92afaec"
format_stamp: "Formatted at 2026-06-21 08:00:54 on dist-test-slave-0pn3"
server_key: "0b66d9082326dc0a6ded38ca7a1be335"
server_key_iv: "b9f35f10d316dd1b45084d2e71d618f1"
server_key_version: "encryptionkey@0"
I20260621 08:00:54.169950 1614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "53bc78cecc9b4da6a21e2e35d92afaec"
format_stamp: "Formatted at 2026-06-21 08:00:54 on dist-test-slave-0pn3"
server_key: "0b66d9082326dc0a6ded38ca7a1be335"
server_key_iv: "b9f35f10d316dd1b45084d2e71d618f1"
server_key_version: "encryptionkey@0"
I20260621 08:00:54.173873 1614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260621 08:00:54.176295 1629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:54.177680 1614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:54.177829 1614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "53bc78cecc9b4da6a21e2e35d92afaec"
format_stamp: "Formatted at 2026-06-21 08:00:54 on dist-test-slave-0pn3"
server_key: "0b66d9082326dc0a6ded38ca7a1be335"
server_key_iv: "b9f35f10d316dd1b45084d2e71d618f1"
server_key_version: "encryptionkey@0"
I20260621 08:00:54.177948 1614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:54.190805 1614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:54.193809 1614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:54.193965 1614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:54.201310 1614 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:44519
I20260621 08:00:54.201321 1681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:44519 every 8 connection(s)
I20260621 08:00:54.202296 1614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:54.205134 1682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:54.208592 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1614
I20260621 08:00:54.208746 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:54.209004 12158 external_mini_cluster.cc:1428] Setting key 214cf322090cf62047c712e05031c91f
I20260621 08:00:54.211282 1682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec: Bootstrap starting.
I20260621 08:00:54.213917 1682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:54.214821 1682 log.cc:824] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:54.217098 1682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec: No bootstrap required, opened a new log
Jun 21 08:00:54 dist-test-slave-0pn3 krb5kdc[1601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:54.220762 1682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44519 } }
I20260621 08:00:54.221007 1682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:54.221096 1682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bc78cecc9b4da6a21e2e35d92afaec, State: Initialized, Role: FOLLOWER
I20260621 08:00:54.221540 1682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44519 } }
I20260621 08:00:54.221692 1682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:54.221769 1682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:54.221874 1682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:54.223031 1682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44519 } }
I20260621 08:00:54.223394 1682 leader_election.cc:302] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53bc78cecc9b4da6a21e2e35d92afaec; no voters:
I20260621 08:00:54.223742 1682 leader_election.cc:288] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:54.223847 1687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:54.224067 1687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [term 1 LEADER]: Becoming Leader. State: Replica: 53bc78cecc9b4da6a21e2e35d92afaec, State: Running, Role: LEADER
I20260621 08:00:54.224088 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:54.210466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33973 (local address 127.11.223.190:44519)
0621 08:00:54.210890 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:54.210906 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:54.210942 (+ 36us) server_negotiation.cc:407] Connection header received
0621 08:00:54.211610 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:54.211629 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:54.211978 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:54.212252 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:54.213236 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:54.214289 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:54.215420 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:54.215954 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:54.216073 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:54.218888 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:54.218892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:54.218906 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:54.218932 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:54.220759 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:54.221297 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:54.221302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:54.221306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:54.221461 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:54.221767 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:54.221771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:54.221774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:54.222296 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:54.222500 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:54.222701 (+ 201us) server_negotiation.cc:299] Negotiation successful
0621 08:00:54.223025 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260621 08:00:54.224845 1687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44519 } }
I20260621 08:00:54.225000 1682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:54.226524 1688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53bc78cecc9b4da6a21e2e35d92afaec. Latest consensus state: current_term: 1 leader_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44519 } } }
I20260621 08:00:54.226646 1688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:54.227464 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:54.228502 1689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bc78cecc9b4da6a21e2e35d92afaec" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 44519 } } }
I20260621 08:00:54.228675 1689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:54.229744 1702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:54.229820 1702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:54.229878 1700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:54.230711 1700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:54.236706 1700 catalog_manager.cc:1347] Generated new cluster ID: 92ba504aba804e27929e605edd7b9929
I20260621 08:00:54.236833 1700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:54.238232 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:54.229624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34667 (local address 127.11.223.190:44519)
0621 08:00:54.229740 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:54.229744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:54.230163 (+ 419us) server_negotiation.cc:407] Connection header received
0621 08:00:54.230309 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:54.230314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:54.230361 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:54.230458 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:54.231159 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:54.231683 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:54.232629 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:54.233050 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:54.233161 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:54.234789 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:54.234792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:54.234795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:54.234819 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:54.236499 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:54.237014 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:54.237017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:54.237018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:54.237101 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:54.237409 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:54.237413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:54.237414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:54.237587 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:54.237728 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:54.237966 (+ 238us) server_negotiation.cc:299] Negotiation successful
0621 08:00:54.238075 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260621 08:00:54.250224 1700 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:54.251140 1700 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:54.261430 1700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53bc78cecc9b4da6a21e2e35d92afaec: Generated new TSK 0
I20260621 08:00:54.262169 1700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:54.266467 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1614
2026-06-21T08:00:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:54 dist-test-slave-0pn3 krb5kdc[1713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:54 dist-test-slave-0pn3 krb5kdc[1713](info): set up 2 sockets
Jun 21 08:00:54 dist-test-slave-0pn3 krb5kdc[1713](info): commencing operation
krb5kdc: starting...
W20260621 08:00:55.292870 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:55 dist-test-slave-0pn3 krb5kdc[1713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:55.413339 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:34413
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:34413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:55.525256 1726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:55.525552 1726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:55.525595 1726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:55.529562 1726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:55.529630 1726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:55.529649 1726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:55.529666 1726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:55.529682 1726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:55.536592 1726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:34413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:34413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:55.537714 1726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:55.538704 1726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:55.547760 1734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:55.547741 1732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:55.547704 1731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:55.547856 1726 server_base.cc:1034] running on GCE node
I20260621 08:00:55.548709 1726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:55.549866 1726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:55.551057 1726 hybrid_clock.cc:648] HybridClock initialized: now 1782028855551046 us; error 32 us; skew 500 ppm
Jun 21 08:00:55 dist-test-slave-0pn3 krb5kdc[1713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028855, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:55.554230 1726 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:55.555210 1726 webserver.cc:460] Webserver started at http://127.11.223.190:42447/ using document root <none> and password file <none>
I20260621 08:00:55.555801 1726 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:55.555891 1726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:55.556109 1726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:55.558368 1726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "278ca726506148e8b4db7429b59d1f91"
format_stamp: "Formatted at 2026-06-21 08:00:55 on dist-test-slave-0pn3"
server_key: "41c7f3e35c6cc435557f4c316b74c17e"
server_key_iv: "a86c7784d23ee2aafad200c3fbe5e148"
server_key_version: "encryptionkey@0"
I20260621 08:00:55.559000 1726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "278ca726506148e8b4db7429b59d1f91"
format_stamp: "Formatted at 2026-06-21 08:00:55 on dist-test-slave-0pn3"
server_key: "41c7f3e35c6cc435557f4c316b74c17e"
server_key_iv: "a86c7784d23ee2aafad200c3fbe5e148"
server_key_version: "encryptionkey@0"
I20260621 08:00:55.562973 1726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260621 08:00:55.565572 1741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:55.566843 1726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:55.566988 1726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "278ca726506148e8b4db7429b59d1f91"
format_stamp: "Formatted at 2026-06-21 08:00:55 on dist-test-slave-0pn3"
server_key: "41c7f3e35c6cc435557f4c316b74c17e"
server_key_iv: "a86c7784d23ee2aafad200c3fbe5e148"
server_key_version: "encryptionkey@0"
I20260621 08:00:55.567104 1726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:55.588088 1726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:55.591104 1726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:55.591353 1726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:55.598881 1726 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:34413
I20260621 08:00:55.598905 1793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:34413 every 8 connection(s)
I20260621 08:00:55.599933 1726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:55.602890 1794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:55.608798 1794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91: Bootstrap starting.
I20260621 08:00:55.610280 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1726
I20260621 08:00:55.610445 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:55.610740 12158 external_mini_cluster.cc:1428] Setting key 6bedd9c97646ee1f7f55661b415eeb54
I20260621 08:00:55.611832 1794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:55.612752 1794 log.cc:824] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:55.614777 1794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91: No bootstrap required, opened a new log
I20260621 08:00:55.618683 1794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278ca726506148e8b4db7429b59d1f91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34413 } }
I20260621 08:00:55.618928 1794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:55.618964 1794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278ca726506148e8b4db7429b59d1f91, State: Initialized, Role: FOLLOWER
I20260621 08:00:55.619542 1794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278ca726506148e8b4db7429b59d1f91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34413 } }
I20260621 08:00:55.619692 1794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:55.619777 1794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:55.619907 1794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:55.620587 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:55.612032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53111 (local address 127.11.223.190:34413)
0621 08:00:55.612537 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:55.612555 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:55.612592 (+ 37us) server_negotiation.cc:407] Connection header received
0621 08:00:55.613448 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:55.613477 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:55.613882 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:55.614235 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:55.614643 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:55.615676 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:55.616819 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:55.617334 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:55.617511 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:55.618426 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:55.618431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:55.618444 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:55.618856 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:55.618943 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:55.618980 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:55.619096 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:55.619218 (+ 122us) server_negotiation.cc:299] Negotiation successful
0621 08:00:55.619556 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":131,"threads_started":1}
I20260621 08:00:55.621027 1794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278ca726506148e8b4db7429b59d1f91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34413 } }
I20260621 08:00:55.621384 1794 leader_election.cc:302] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 278ca726506148e8b4db7429b59d1f91; no voters:
I20260621 08:00:55.621762 1794 leader_election.cc:288] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:55.621860 1799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:55.622105 1799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [term 1 LEADER]: Becoming Leader. State: Replica: 278ca726506148e8b4db7429b59d1f91, State: Running, Role: LEADER
I20260621 08:00:55.622471 1799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278ca726506148e8b4db7429b59d1f91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34413 } }
I20260621 08:00:55.623126 1794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:55.623692 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:55.624454 1801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 278ca726506148e8b4db7429b59d1f91. Latest consensus state: current_term: 1 leader_uuid: "278ca726506148e8b4db7429b59d1f91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278ca726506148e8b4db7429b59d1f91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34413 } } }
I20260621 08:00:55.624454 1800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "278ca726506148e8b4db7429b59d1f91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278ca726506148e8b4db7429b59d1f91" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 34413 } } }
I20260621 08:00:55.624639 1801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:55.624639 1800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91 [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:55.628443 1818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:55.628535 1818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:55.628640 1809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:55.629422 1809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:55.631130 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:55.626246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39323 (local address 127.11.223.190:34413)
0621 08:00:55.626368 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:55.626372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:55.626621 (+ 249us) server_negotiation.cc:407] Connection header received
0621 08:00:55.626806 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:55.626823 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:55.626875 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:55.626929 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:55.627349 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:55.628098 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:55.629288 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:55.629622 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:55.629732 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:55.630652 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:55.630656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:55.630659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:55.630673 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:55.630703 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:55.630716 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:55.630797 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:55.630923 (+ 126us) server_negotiation.cc:299] Negotiation successful
0621 08:00:55.630985 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260621 08:00:55.635506 1809 catalog_manager.cc:1347] Generated new cluster ID: bfe4c45f6eb64616a50a16787142471c
I20260621 08:00:55.635632 1809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:55.647312 1809 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:55.648232 1809 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:55.656188 1809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 278ca726506148e8b4db7429b59d1f91: Generated new TSK 0
I20260621 08:00:55.656911 1809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:55.661028 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1726
2026-06-21T08:00:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:55 dist-test-slave-0pn3 krb5kdc[1825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:55 dist-test-slave-0pn3 krb5kdc[1825](info): set up 2 sockets
Jun 21 08:00:55 dist-test-slave-0pn3 krb5kdc[1825](info): commencing operation
krb5kdc: starting...
W20260621 08:00:56.696776 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:56 dist-test-slave-0pn3 krb5kdc[1825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:56.813019 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45531
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:56.922300 1838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:56.922552 1838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:56.922595 1838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:56.926586 1838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:56.926646 1838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:56.926666 1838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:56.926717 1838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:56.926752 1838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:56.931924 1838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:56.932991 1838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:56.934060 1838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:56.942399 1843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:56.942445 1838 server_base.cc:1034] running on GCE node
W20260621 08:00:56.942395 1846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:56.942394 1844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:56.943125 1838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:56.944182 1838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:56.945336 1838 hybrid_clock.cc:648] HybridClock initialized: now 1782028856945305 us; error 45 us; skew 500 ppm
Jun 21 08:00:56 dist-test-slave-0pn3 krb5kdc[1825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028856, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:56.948037 1838 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:56.948933 1838 webserver.cc:460] Webserver started at http://127.11.223.190:40001/ using document root <none> and password file <none>
I20260621 08:00:56.949450 1838 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:56.949533 1838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:56.949749 1838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:56.951642 1838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "63eefc5d8ace4245992ceeec1385dbe0"
format_stamp: "Formatted at 2026-06-21 08:00:56 on dist-test-slave-0pn3"
server_key: "d9728d31c2de30ef6c70eb36e4d04bbb"
server_key_iv: "a6c22d07cf48d3430ce2ba5398ee8e28"
server_key_version: "encryptionkey@0"
I20260621 08:00:56.952186 1838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "63eefc5d8ace4245992ceeec1385dbe0"
format_stamp: "Formatted at 2026-06-21 08:00:56 on dist-test-slave-0pn3"
server_key: "d9728d31c2de30ef6c70eb36e4d04bbb"
server_key_iv: "a6c22d07cf48d3430ce2ba5398ee8e28"
server_key_version: "encryptionkey@0"
I20260621 08:00:56.955638 1838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260621 08:00:56.957924 1853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:56.959065 1838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260621 08:00:56.959159 1838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "63eefc5d8ace4245992ceeec1385dbe0"
format_stamp: "Formatted at 2026-06-21 08:00:56 on dist-test-slave-0pn3"
server_key: "d9728d31c2de30ef6c70eb36e4d04bbb"
server_key_iv: "a6c22d07cf48d3430ce2ba5398ee8e28"
server_key_version: "encryptionkey@0"
I20260621 08:00:56.959235 1838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:56.963482 1838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:56.966563 1838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:56.966699 1838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:56.973258 1838 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:45531
I20260621 08:00:56.973307 1905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:45531 every 8 connection(s)
I20260621 08:00:56.974171 1838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:56.976935 1906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:56.978619 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1838
I20260621 08:00:56.978755 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:56.979007 12158 external_mini_cluster.cc:1428] Setting key f358a71be8f41ac5465ac11ccefa6191
I20260621 08:00:56.982952 1906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0: Bootstrap starting.
I20260621 08:00:56.983858 1909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:56.980273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34285 (local address 127.11.223.190:45531)
0621 08:00:56.980689 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:56.980706 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:56.980739 (+ 33us) server_negotiation.cc:407] Connection header received
0621 08:00:56.981394 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:56.981412 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:56.981669 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:56.981981 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:56.982750 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:56.982755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:56.982766 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:56.982775 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:56.982824 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:56.982842 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:56.982979 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:56.983045 (+ 66us) server_negotiation.cc:299] Negotiation successful
0621 08:00:56.983190 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
I20260621 08:00:56.985176 1906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:56.985976 1906 log.cc:824] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:56.986516 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:56.987826 1906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0: No bootstrap required, opened a new log
I20260621 08:00:56.990026 1909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:56.988563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43977 (local address 127.11.223.190:45531)
0621 08:00:56.988693 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:56.988698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:56.988853 (+ 155us) server_negotiation.cc:407] Connection header received
0621 08:00:56.988945 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:56.988947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:56.988978 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:56.989049 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:56.989680 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:56.989685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:56.989691 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0621 08:00:56.989703 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:56.989728 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0621 08:00:56.989747 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:56.989848 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:56.989882 (+ 34us) server_negotiation.cc:299] Negotiation successful
0621 08:00:56.989915 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260621 08:00:56.991983 1906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63eefc5d8ace4245992ceeec1385dbe0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45531 } }
I20260621 08:00:56.992192 1906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260621 08:00:56.992224 1906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63eefc5d8ace4245992ceeec1385dbe0, State: Initialized, Role: FOLLOWER
I20260621 08:00:56.992792 1906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63eefc5d8ace4245992ceeec1385dbe0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45531 } }
I20260621 08:00:56.992964 1906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260621 08:00:56.993033 1906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260621 08:00:56.993127 1906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:56.994030 1906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63eefc5d8ace4245992ceeec1385dbe0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45531 } }
I20260621 08:00:56.994375 1906 leader_election.cc:302] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63eefc5d8ace4245992ceeec1385dbe0; no voters:
I20260621 08:00:56.994693 1906 leader_election.cc:288] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260621 08:00:56.994808 1916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:56.995023 1916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [term 1 LEADER]: Becoming Leader. State: Replica: 63eefc5d8ace4245992ceeec1385dbe0, State: Running, Role: LEADER
I20260621 08:00:56.995409 1916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63eefc5d8ace4245992ceeec1385dbe0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45531 } }
I20260621 08:00:56.995934 1906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:56.997231 1917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63eefc5d8ace4245992ceeec1385dbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63eefc5d8ace4245992ceeec1385dbe0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45531 } } }
I20260621 08:00:56.997258 1918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63eefc5d8ace4245992ceeec1385dbe0. Latest consensus state: current_term: 1 leader_uuid: "63eefc5d8ace4245992ceeec1385dbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63eefc5d8ace4245992ceeec1385dbe0" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 45531 } } }
I20260621 08:00:56.997372 1917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:56.997375 1918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:56.997738 1925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:57.001211 1925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:57.006456 1925 catalog_manager.cc:1347] Generated new cluster ID: 828bbe45531043e7a62c93017da1ccaa
I20260621 08:00:57.006527 1925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:57.017620 1925 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:57.017918 1925 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:57.021842 1925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63eefc5d8ace4245992ceeec1385dbe0: Generated new TSK 0
I20260621 08:00:57.022446 1925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260621 08:00:57.030995 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1838
2026-06-21T08:00:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1370 ms)
[----------] 18 tests from AuthTokenIssuingTest (24970 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:00:57 dist-test-slave-0pn3 krb5kdc[1937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:00:57 dist-test-slave-0pn3 krb5kdc[1937](info): set up 2 sockets
Jun 21 08:00:57 dist-test-slave-0pn3 krb5kdc[1937](info): commencing operation
krb5kdc: starting...
W20260621 08:00:58.068871 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:00:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:00:58.187276 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43749
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43749,127.11.223.189:35745,127.11.223.188:46425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:00:58.298115 1950 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 08:00:58.298480 1950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:58.298528 1950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:58.298601 1950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:58.302474 1950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:58.302584 1950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:58.302621 1950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:58.302657 1950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:58.302691 1950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:58.307910 1950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43749,127.11.223.189:35745,127.11.223.188:46425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:43749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:58.308996 1950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:58.310060 1950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:58.318825 1958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:58.318842 1955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:58.318845 1956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:58.319165 1950 server_base.cc:1034] running on GCE node
I20260621 08:00:58.319806 1950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:58.321017 1950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:58.322243 1950 hybrid_clock.cc:648] HybridClock initialized: now 1782028858322223 us; error 29 us; skew 500 ppm
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:58.325059 1950 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:00:58.325969 1950 webserver.cc:460] Webserver started at http://127.11.223.190:37763/ using document root <none> and password file <none>
I20260621 08:00:58.326552 1950 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:58.326642 1950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:58.326875 1950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:58.329089 1950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "2c37d120c9d949ea858e096d68b19e80"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "781bee6ce770f17af618e98ef7568a7d"
server_key_iv: "e4ed2d57d1bef8c18cc63737791c711c"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.329691 1950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "2c37d120c9d949ea858e096d68b19e80"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "781bee6ce770f17af618e98ef7568a7d"
server_key_iv: "e4ed2d57d1bef8c18cc63737791c711c"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.333643 1950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260621 08:00:58.336036 1965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:58.337320 1950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 08:00:58.337481 1950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "2c37d120c9d949ea858e096d68b19e80"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "781bee6ce770f17af618e98ef7568a7d"
server_key_iv: "e4ed2d57d1bef8c18cc63737791c711c"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.337594 1950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:58.353808 1950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:58.356693 1950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:58.356889 1950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:58.363873 1950 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:43749
I20260621 08:00:58.364133 2017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:43749 every 8 connection(s)
I20260621 08:00:58.364914 1950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:00:58.367641 2018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:58.371613 2018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.373750 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 1950
I20260621 08:00:58.373906 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:00:58.374193 12158 external_mini_cluster.cc:1428] Setting key 5231c446cd5adb50dc32c3a4dd7ca057
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.189@KRBTEST.COM" created.
I20260621 08:00:58.387089 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.374058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43749 (local address 127.0.0.1:38822)
0621 08:00:58.374680 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.374729 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.375595 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.376290 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.376296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.376773 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.377024 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.377037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.378184 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.378187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.378914 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.378923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.379575 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.379579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.379737 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.380435 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.380450 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.381922 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.383903 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.383916 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.383930 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.384209 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.384621 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.384623 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.384624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.384754 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.385547 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.385556 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.385745 (+ 189us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.385955 (+ 210us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.386365 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":61,"spinlock_wait_cycles":14720,"thread_start_us":169,"threads_started":1}
I20260621 08:00:58.387096 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.374058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.11.223.190:43749)
0621 08:00:58.374550 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.374574 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.374859 (+ 285us) server_negotiation.cc:407] Connection header received
0621 08:00:58.375913 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.375917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.376102 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.376297 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.377177 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.378046 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.379037 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.379421 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.379589 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.382071 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.382076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.382087 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.382107 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.383777 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.384351 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.384360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.384365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.384510 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.384849 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.384855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.384859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.385410 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.385571 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.386017 (+ 446us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.386259 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":318,"thread_start_us":136,"threads_started":1}
I20260621 08:00:58.389822 2018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.390637 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.390288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43632)
0621 08:00:58.390440 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.390543 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260621 08:00:58.390856 1968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
I20260621 08:00:58.391717 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.391467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43642)
0621 08:00:58.391592 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.391646 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260621 08:00:58.391873 2018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:35745: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab.
Entry for principal kudu/127.11.223.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab.
I20260621 08:00:58.399158 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:35745
--webserver_interface=127.11.223.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab
--principal=kudu/127.11.223.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43749,127.11.223.189:35745,127.11.223.188:46425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:00:58.441129 2018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } attempt: 1
I20260621 08:00:58.442130 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.441773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43646)
0621 08:00:58.441935 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.442038 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260621 08:00:58.442924 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.442670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43660)
0621 08:00:58.442799 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.442858 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260621 08:00:58.443087 2018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:35745: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
W20260621 08:00:58.510565 2025 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 08:00:58.510943 2025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:58.511044 2025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:58.511137 2025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260621 08:00:58.511296 2018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } attempt: 2
I20260621 08:00:58.512195 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.511833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43672)
0621 08:00:58.512020 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.512094 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260621 08:00:58.513020 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.512793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43674)
0621 08:00:58.512903 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.512955 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260621 08:00:58.513190 2018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:35745: Network error: Client connection negotiation failed: client connection to 127.11.223.189:35745: connect: Connection refused (error 111)
W20260621 08:00:58.515829 2025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:58.515882 2025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:58.515902 2025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:58.515959 2025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:58.515991 2025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:58.521517 2025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43749,127.11.223.189:35745,127.11.223.188:46425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab
--principal=kudu/127.11.223.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:35745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.223.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:58.522662 2025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:58.523824 2025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:58.532450 2033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:58.532452 2030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:58.532490 2025 server_base.cc:1034] running on GCE node
W20260621 08:00:58.532452 2031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:58.533317 2025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:58.534435 2025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:58.535604 2025 hybrid_clock.cc:648] HybridClock initialized: now 1782028858535582 us; error 30 us; skew 500 ppm
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:58.538455 2025 init.cc:377] Logged in from keytab as kudu/127.11.223.189@KRBTEST.COM (short username kudu)
I20260621 08:00:58.539314 2025 webserver.cc:460] Webserver started at http://127.11.223.189:32835/ using document root <none> and password file <none>
I20260621 08:00:58.539840 2025 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:58.539898 2025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:58.540077 2025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:58.542284 2025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data/instance:
uuid: "0bb66cfae9d64cf88698bd08d111fb8a"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "1262b5545ee8bf1d9928fc16138204d0"
server_key_iv: "d2df6ef0a715500e892f55d1b42db8ba"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.542836 2025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal/instance:
uuid: "0bb66cfae9d64cf88698bd08d111fb8a"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "1262b5545ee8bf1d9928fc16138204d0"
server_key_iv: "d2df6ef0a715500e892f55d1b42db8ba"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.547662 2025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260621 08:00:58.550302 2040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:58.551545 2025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:00:58.551687 2025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal
uuid: "0bb66cfae9d64cf88698bd08d111fb8a"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "1262b5545ee8bf1d9928fc16138204d0"
server_key_iv: "d2df6ef0a715500e892f55d1b42db8ba"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.551803 2025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:58.585112 2025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:58.588701 2025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:58.588930 2025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:58.595928 2025 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.189:35745
I20260621 08:00:58.596221 2092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.189:35745 every 8 connection(s)
I20260621 08:00:58.597059 2025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
I20260621 08:00:58.599895 2093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:58.603760 2093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.604393 2018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } attempt: 3
I20260621 08:00:58.606302 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 2025
I20260621 08:00:58.606436 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-1/wal/instance
I20260621 08:00:58.606719 12158 external_mini_cluster.cc:1428] Setting key 38489f7e74c29537b302d63c39a82efa
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.223.188@KRBTEST.COM; defaulting to no policy
I20260621 08:00:58.618305 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.604956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43676)
0621 08:00:58.605134 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.605153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.605257 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.607547 (+ 2290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.607550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.607756 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.607887 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.607898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.609348 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.609354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.610217 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.610246 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.611202 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.611208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.611365 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.611997 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.612017 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.613711 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.616013 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.616016 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.616019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.616474 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.616888 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.616892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.616894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.616992 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.617772 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.617775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.617875 (+ 100us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.617986 (+ 111us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.618139 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260621 08:00:58.618466 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.606290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.11.223.190:43749)
0621 08:00:58.606406 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.606411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.606856 (+ 445us) server_negotiation.cc:407] Connection header received
0621 08:00:58.607291 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.607295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.607349 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.607423 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.608557 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.609392 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.610969 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.611950 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.612075 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.614831 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.614837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.614842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.614872 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.616531 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.617058 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.617062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.617064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.617141 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.617406 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.617410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.617411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.617623 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.617732 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.618148 (+ 416us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.618276 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.188@KRBTEST.COM" created.
I20260621 08:00:58.619472 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.605577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43676 (local address 127.11.223.189:35745)
0621 08:00:58.605994 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.606031 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.606072 (+ 41us) server_negotiation.cc:407] Connection header received
0621 08:00:58.607046 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.607070 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.607350 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.607581 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.608027 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.609212 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.610380 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.611062 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.611241 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.613847 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.613854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.613871 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.613892 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.615900 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.616611 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.616620 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.616629 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.616777 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.617080 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.617085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.617086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.617675 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.617797 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.618094 (+ 297us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.618410 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":287,"thread_start_us":128,"threads_started":1}
I20260621 08:00:58.619983 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.606230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43749 (local address 127.0.0.1:38834)
0621 08:00:58.606722 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.606772 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.607107 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.607471 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.607480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.607928 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.608370 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.608390 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.609539 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.609542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.610740 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.610754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.612133 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.612138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.612271 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.612841 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.612858 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.614636 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.616640 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.616648 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.616652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.616931 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.617229 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.617231 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.617232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.617311 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.617754 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.617761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.617907 (+ 146us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.618080 (+ 173us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.619862 (+ 1782us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":1529,"spinlock_wait_cycles":4480,"thread_start_us":91,"threads_started":1}
I20260621 08:00:58.622284 2093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.622740 2018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.623600 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.623341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51664)
0621 08:00:58.623477 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.623532 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260621 08:00:58.624480 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.624075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51668)
0621 08:00:58.624271 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.624392 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260621 08:00:58.624691 2018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:46425: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 08:00:58.631232 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.622842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43692)
0621 08:00:58.623055 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.623072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.623156 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.623602 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.623606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.623807 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.623957 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.623967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.624947 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.624954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.625897 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.625907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.626385 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.626390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.626503 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.626959 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.626975 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.628371 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.629976 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.629979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.629980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.630154 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.630457 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.630459 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.630460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.630530 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.630843 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.630845 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.630918 (+ 73us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.630992 (+ 74us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.631101 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260621 08:00:58.631321 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.622866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.11.223.189:35745)
0621 08:00:58.623306 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.623312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.623324 (+ 12us) server_negotiation.cc:407] Connection header received
0621 08:00:58.623357 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.623360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.623403 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.623493 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.624069 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.624829 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.626024 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.626263 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.626339 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.628490 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.628493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.628495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.628515 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.629869 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.630242 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.630244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.630246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.630373 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.630602 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.630604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.630605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.630769 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.630845 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.631034 (+ 189us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.631157 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":358}
I20260621 08:00:58.632030 2093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.632839 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.632609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51680)
0621 08:00:58.632716 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.632791 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260621 08:00:58.633394 2041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
I20260621 08:00:58.634203 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.634005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51694)
0621 08:00:58.634105 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.634152 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260621 08:00:58.634363 2093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:46425: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab.
Entry for principal kudu/127.11.223.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab.
I20260621 08:00:58.635650 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:46425
--webserver_interface=127.11.223.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab
--principal=kudu/127.11.223.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:43749,127.11.223.189:35745,127.11.223.188:46425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:00:58.655857 2018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } attempt: 1
I20260621 08:00:58.656826 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.656528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51708)
0621 08:00:58.656671 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.656743 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260621 08:00:58.657574 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.657363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51710)
0621 08:00:58.657468 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.657520 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260621 08:00:58.657739 2018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:46425: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
I20260621 08:00:58.683604 2093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } attempt: 1
I20260621 08:00:58.684588 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.684182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51712)
0621 08:00:58.684399 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.684480 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260621 08:00:58.685367 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.685117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51718)
0621 08:00:58.685232 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.685289 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260621 08:00:58.685527 2093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:46425: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
I20260621 08:00:58.732889 2018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } attempt: 2
I20260621 08:00:58.733714 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.733421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51730)
0621 08:00:58.733559 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.733625 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260621 08:00:58.734414 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.734210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51746)
0621 08:00:58.734313 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.734361 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260621 08:00:58.734549 2018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:46425: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
W20260621 08:00:58.748901 2100 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 08:00:58.749253 2100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:00:58.749307 2100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:00:58.749382 2100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:00:58.753361 2100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:00:58.753432 2100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:00:58.753464 2100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:00:58.753509 2100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:00:58.753543 2100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:00:58.753716 2093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } attempt: 2
I20260621 08:00:58.754513 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.754190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51754)
0621 08:00:58.754357 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.754427 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260621 08:00:58.755254 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.754999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51770)
0621 08:00:58.755139 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.755190 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260621 08:00:58.755416 2093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:46425: Network error: Client connection negotiation failed: client connection to 127.11.223.188:46425: connect: Connection refused (error 111)
I20260621 08:00:58.759227 2100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:43749,127.11.223.189:35745,127.11.223.188:46425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab
--principal=kudu/127.11.223.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:46425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.223.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:00:58.760275 2100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:00:58.761379 2100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:00:58.769721 2108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:00:58.769739 2106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:58.769783 2100 server_base.cc:1034] running on GCE node
W20260621 08:00:58.769729 2105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:00:58.770431 2100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:00:58.771548 2100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:00:58.772728 2100 hybrid_clock.cc:648] HybridClock initialized: now 1782028858772690 us; error 50 us; skew 500 ppm
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:00:58.775324 2100 init.cc:377] Logged in from keytab as kudu/127.11.223.188@KRBTEST.COM (short username kudu)
I20260621 08:00:58.776122 2100 webserver.cc:460] Webserver started at http://127.11.223.188:34537/ using document root <none> and password file <none>
I20260621 08:00:58.776697 2100 fs_manager.cc:359] Metadata directory not provided
I20260621 08:00:58.776755 2100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:00:58.776917 2100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:00:58.778826 2100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data/instance:
uuid: "611435b40f0c464a8107ed7f65483d25"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "0ebe7ea6895740a79bf3c5cab1a3fd11"
server_key_iv: "6d17096b5d4ff2a6992e14b15d856b47"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.779330 2100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal/instance:
uuid: "611435b40f0c464a8107ed7f65483d25"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "0ebe7ea6895740a79bf3c5cab1a3fd11"
server_key_iv: "6d17096b5d4ff2a6992e14b15d856b47"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.782737 2100 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260621 08:00:58.784910 2115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:00:58.786067 2100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260621 08:00:58.786204 2100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal
uuid: "611435b40f0c464a8107ed7f65483d25"
format_stamp: "Formatted at 2026-06-21 08:00:58 on dist-test-slave-0pn3"
server_key: "0ebe7ea6895740a79bf3c5cab1a3fd11"
server_key_iv: "6d17096b5d4ff2a6992e14b15d856b47"
server_key_version: "encryptionkey@0"
I20260621 08:00:58.786320 2100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:00:58.801157 2100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:00:58.803910 2100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:00:58.804121 2100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:00:58.810897 2100 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.188:46425
I20260621 08:00:58.811174 2167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.188:46425 every 8 connection(s)
I20260621 08:00:58.811895 2100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
I20260621 08:00:58.814656 2168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:00:58.818352 2168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.821764 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 2100
I20260621 08:00:58.821897 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1782028587408853-12158-0/minicluster-data/master-2/wal/instance
I20260621 08:00:58.822190 12158 external_mini_cluster.cc:1428] Setting key 2494548ca37d6a8db1d9efe09b89d73b
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
I20260621 08:00:58.832526 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.820131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.11.223.190:43749)
0621 08:00:58.820291 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.820294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.821153 (+ 859us) server_negotiation.cc:407] Connection header received
0621 08:00:58.822411 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.822414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.822461 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.822531 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.824100 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.824673 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.825786 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.826023 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.826141 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.828903 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.828910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.828915 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.828951 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.830574 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.831049 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.831053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.831055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.831146 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.831464 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.831468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.831470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.831664 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.831767 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.832203 (+ 436us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.832353 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260621 08:00:58.833330 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.820556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:43749 (local address 127.0.0.1:38848)
0621 08:00:58.820993 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.821036 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.822180 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.822718 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.822744 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.823304 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.823959 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.823976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.824783 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.824788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.825663 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.825672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.826174 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.826177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.826361 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.826948 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.826963 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.828744 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.830699 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.830707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.830717 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.830949 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.831233 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.831238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.831240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.831374 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.831773 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.831781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.831997 (+ 216us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.832161 (+ 164us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.832472 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":50,"spinlock_wait_cycles":1280,"thread_start_us":111,"threads_started":1}
I20260621 08:00:58.833333 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.822908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.11.223.188:46425)
0621 08:00:58.823202 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.823214 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.823426 (+ 212us) server_negotiation.cc:407] Connection header received
0621 08:00:58.823520 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.823525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.823730 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.823894 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.824607 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.825479 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.826300 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.826643 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.826757 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.829307 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.829313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.829330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.829349 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.830833 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.831242 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.831248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.831249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.831384 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.831645 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.831650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.831651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.831983 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.832182 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.832393 (+ 211us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.832661 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":190,"spinlock_wait_cycles":9472,"thread_start_us":104,"threads_started":1}
I20260621 08:00:58.833717 2018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } attempt: 3
I20260621 08:00:58.834823 2168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:00:58.844668 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.834088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51774)
0621 08:00:58.834225 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.834239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.834309 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.834789 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.834792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.834954 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.835043 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.835049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.836543 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.836547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.837375 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.837384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.837852 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.837856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.837939 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.838412 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.838425 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.839893 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.842932 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.842936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.842938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.843255 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.843578 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.843583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.843588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.843690 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.844128 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.844131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.844226 (+ 95us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.844349 (+ 123us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.844480 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":1}
I20260621 08:00:58.844842 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.834263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.11.223.188:46425)
0621 08:00:58.834533 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.834541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.834554 (+ 13us) server_negotiation.cc:407] Connection header received
0621 08:00:58.834617 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.834619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.834658 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.835146 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.835237 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.836459 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.837492 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.837746 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.837849 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.840038 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.840042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.840044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.840070 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.842825 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.843370 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.843374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.843376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.843472 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.843791 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.843794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.843796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.844006 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.844131 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.844432 (+ 301us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.844539 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":112,"threads_started":1}
I20260621 08:00:58.845636 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.836123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.11.223.190:43749)
0621 08:00:58.836212 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.836215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.836358 (+ 143us) server_negotiation.cc:407] Connection header received
0621 08:00:58.836458 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.836461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.836509 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.836582 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.837283 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.837985 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.839047 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.839264 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.839367 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.841879 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.841882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.841886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.841910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.843735 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.844201 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.844204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.844206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.844278 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.844606 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.844611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.844612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.844831 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.845043 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.845386 (+ 343us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.845471 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260621 08:00:58.846621 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.835382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.11.223.189:35745)
0621 08:00:58.835530 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.835533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.835543 (+ 10us) server_negotiation.cc:407] Connection header received
0621 08:00:58.835577 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.835579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.835612 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.835702 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.836033 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.836772 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.837808 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.838159 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.838265 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.841756 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.841759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.841762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.841787 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.843518 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.843970 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.843974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.843975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.844060 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.845109 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.845112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.845114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.845723 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.845826 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.846368 (+ 542us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.846480 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260621 08:00:58.846720 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.835270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:35745 (local address 127.0.0.1:43694)
0621 08:00:58.835353 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.835365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.835428 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.835710 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.835713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.835858 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.835928 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.835932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.836869 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.836875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.837680 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.837690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.838400 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.838403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.838483 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.838880 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.838892 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.841619 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.843641 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.843643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.843644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.843873 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.844892 (+ 1019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.844898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.844900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.845004 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.845868 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.845871 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.846187 (+ 316us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.846314 (+ 127us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.846432 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260621 08:00:58.846594 2093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } attempt: 3
I20260621 08:00:58.847584 2168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:00:58.848560 2018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80: Bootstrap starting.
I20260621 08:00:58.851521 2018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:58.852502 2018 log.cc:824] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80: Log is configured to *not* fsync() on all Append() calls
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 08:00:58 dist-test-slave-0pn3 krb5kdc[1937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 08:00:58.854517 2018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80: No bootstrap required, opened a new log
I20260621 08:00:58.857968 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.848026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51794)
0621 08:00:58.848246 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.848258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.848351 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.848748 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.848751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.848895 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.848974 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.848980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.849967 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.849970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.850700 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.850710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.851301 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.851304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.851406 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.851844 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.851856 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.853290 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.855265 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.855280 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.855282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.855912 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.856207 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.856213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.856219 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.856336 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.856869 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.856873 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.856972 (+ 99us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.857062 (+ 90us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.857798 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260621 08:00:58.858000 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.848167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51794 (local address 127.11.223.188:46425)
0621 08:00:58.848570 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.848574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.848585 (+ 11us) server_negotiation.cc:407] Connection header received
0621 08:00:58.848612 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.848615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.848647 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.848727 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.849086 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.849801 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.850830 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.851188 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.851706 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.853428 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.853432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.853435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.853459 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.855164 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.856012 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.856017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.856020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.856100 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.856451 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.856454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.856457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.856754 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.857230 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.857317 (+ 87us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.857858 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":435,"server-negotiator.queue_time_us":340}
I20260621 08:00:58.858914 2018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.859150 2018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c37d120c9d949ea858e096d68b19e80, State: Initialized, Role: FOLLOWER
I20260621 08:00:58.859496 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.846965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.11.223.189:35745)
0621 08:00:58.847151 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.847156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.847237 (+ 81us) server_negotiation.cc:407] Connection header received
0621 08:00:58.847277 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.847281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.847332 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.847404 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.848286 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.848974 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.850016 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.850351 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.850416 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.855190 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.855194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.855197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.855221 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.856861 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.857357 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.857360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.857362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.857444 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.857757 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.857761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.857762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.857961 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.858184 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.859039 (+ 855us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.859171 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":115,"threads_started":1}
I20260621 08:00:58.859488 2018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.860620 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.847414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:46425 (local address 127.0.0.1:51790)
0621 08:00:58.847642 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0621 08:00:58.847655 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 08:00:58.847743 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:00:58.848176 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:00:58.848179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:00:58.848361 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:00:58.848436 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.848442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.849724 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.849727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.850488 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:00:58.850499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.851155 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.851161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:00:58.851258 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.851684 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:00:58.851700 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:00:58.854537 (+ 2837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:00:58.858660 (+ 4123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.858663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.858665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.858910 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.859228 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:00:58.859231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:00:58.859233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:00:58.859334 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:00:58.860166 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:00:58.860172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:00:58.860271 (+ 99us) client_negotiation.cc:769] Sending connection context
0621 08:00:58.860373 (+ 102us) client_negotiation.cc:240] Negotiation successful
0621 08:00:58.860493 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260621 08:00:58.860975 2018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:58.861193 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.847866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.11.223.188:46425)
0621 08:00:58.847967 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.847970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.847982 (+ 12us) server_negotiation.cc:407] Connection header received
0621 08:00:58.848016 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.848019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.848058 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.848796 (+ 738us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.848843 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.849577 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.850618 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.851016 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.851468 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.855246 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.855253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.855258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.855280 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.857683 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.859033 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.859036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.859038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.859117 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.859867 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.859870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.859871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.860055 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.860869 (+ 814us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.860955 (+ 86us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.861059 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":32}
I20260621 08:00:58.861220 2168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: Bootstrap starting.
I20260621 08:00:58.862843 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:00:58.863682 2168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:58.864585 2168 log.cc:824] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:58.864554 1992 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2c37d120c9d949ea858e096d68b19e80"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38854
I20260621 08:00:58.864813 1992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260621 08:00:58.864923 1992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:58.866207 1992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
W20260621 08:00:58.866566 2187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:58.866694 2187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:58.866817 2168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: No bootstrap required, opened a new log
I20260621 08:00:58.867148 1992 leader_election.cc:288] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [CANDIDATE]: Term 1 election: Requested vote from peers 0bb66cfae9d64cf88698bd08d111fb8a (127.11.223.189:35745), 611435b40f0c464a8107ed7f65483d25 (127.11.223.188:46425)
I20260621 08:00:58.867661 2093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: Bootstrap starting.
I20260621 08:00:58.871937 2093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: Neither blocks nor log segments found. Creating new log.
I20260621 08:00:58.868942 1950 master_runner.cc:186] Error getting master registration for 127.11.223.189:35745: OK, instance_id { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" instance_seqno: 1782028858593920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 08:00:58.872220 1950 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 08:00:58.872406 2168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.872766 2168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611435b40f0c464a8107ed7f65483d25, State: Initialized, Role: FOLLOWER
I20260621 08:00:58.872785 2093 log.cc:824] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: Log is configured to *not* fsync() on all Append() calls
I20260621 08:00:58.873265 2168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.874420 2195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.874712 2195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.874751 2093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: No bootstrap required, opened a new log
I20260621 08:00:58.875126 2142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c37d120c9d949ea858e096d68b19e80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "611435b40f0c464a8107ed7f65483d25"
I20260621 08:00:58.875348 2142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:58.876662 2142 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c37d120c9d949ea858e096d68b19e80 in term 1.
I20260621 08:00:58.875054 2168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:58.877113 1966 leader_election.cc:302] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c37d120c9d949ea858e096d68b19e80, 611435b40f0c464a8107ed7f65483d25; no voters:
I20260621 08:00:58.877712 2196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.878010 2196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.878249 2196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:00:58.878952 2196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [term 1 LEADER]: Becoming Leader. State: Replica: 2c37d120c9d949ea858e096d68b19e80, State: Running, Role: LEADER
I20260621 08:00:58.879015 2093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.879249 2093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb66cfae9d64cf88698bd08d111fb8a, State: Initialized, Role: FOLLOWER
I20260621 08:00:58.879729 2093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.879881 2196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } }
I20260621 08:00:58.881181 2093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:00:58.881382 2067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c37d120c9d949ea858e096d68b19e80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0bb66cfae9d64cf88698bd08d111fb8a"
I20260621 08:00:58.881587 2067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:00:58.882871 2067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c37d120c9d949ea858e096d68b19e80 in term 1.
I20260621 08:00:58.885401 2176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c37d120c9d949ea858e096d68b19e80. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.885529 2176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: This master's current role is: LEADER
W20260621 08:00:58.887432 2207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:58.887516 2207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260621 08:00:58.888778 2222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:00:58.888890 2222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:00:58.889225 2224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:00:58.890362 2224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:00:58.893424 2209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.893563 2209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.899408 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.879752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.11.223.189:35745)
0621 08:00:58.879854 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.879860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.888411 (+ 8551us) server_negotiation.cc:407] Connection header received
0621 08:00:58.890604 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.890607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.890652 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.890716 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.891623 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.892309 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.893462 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.893789 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.893861 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.895755 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.895759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.895761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.895785 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.897496 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.898097 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.898101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.898103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.898201 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.898499 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.898502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.898504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.898696 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.898790 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.899128 (+ 338us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.899256 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260621 08:00:58.899791 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.879579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.11.223.190:43749)
0621 08:00:58.879686 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.879690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.885047 (+ 5357us) server_negotiation.cc:407] Connection header received
0621 08:00:58.885217 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.885222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.885276 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.885333 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.888576 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.889252 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.892575 (+ 3323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.892943 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.893024 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.895281 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.895287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.895292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.895323 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.897553 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.898138 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.898142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.898144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.898251 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.898559 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.898564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.898568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.898797 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.898910 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.899506 (+ 596us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.899635 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260621 08:00:58.901634 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:00:58.879231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51806 (local address 127.11.223.188:46425)
0621 08:00:58.879352 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:58.879357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:58.883484 (+ 4127us) server_negotiation.cc:407] Connection header received
0621 08:00:58.883692 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:58.883696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:58.883742 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:58.883792 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:58.886328 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.887056 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.891653 (+ 4597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:00:58.891972 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:00:58.892048 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:00:58.894089 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:00:58.894094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:00:58.894098 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:00:58.894125 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:00:58.898047 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.898525 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.898530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.898532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.898615 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:00:58.898897 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:00:58.898900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:00:58.898902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:00:58.899097 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:00:58.899266 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0621 08:00:58.901378 (+ 2112us) server_negotiation.cc:299] Negotiation successful
0621 08:00:58.901490 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260621 08:00:58.912684 2142 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [term 1 FOLLOWER]: Refusing update from remote peer 2c37d120c9d949ea858e096d68b19e80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:58.914876 2176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:58.916198 2067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [term 1 FOLLOWER]: Refusing update from remote peer 2c37d120c9d949ea858e096d68b19e80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:00:58.916824 2176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:00:58.920401 2195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c37d120c9d949ea858e096d68b19e80. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.920516 2195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.921859 2209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c37d120c9d949ea858e096d68b19e80. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.921998 2209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.926205 2176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.926321 2176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:58.927482 2209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.927579 2209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.927981 2227 mvcc.cc:205] Tried to move back new op lower bound from 7299190206098010112 to 7299190205986926592. Current Snapshot: MvccSnapshot[applied={T|T < 7299190206098010112}]
I20260621 08:00:58.928401 2229 mvcc.cc:205] Tried to move back new op lower bound from 7299190206098010112 to 7299190205986926592. Current Snapshot: MvccSnapshot[applied={T|T < 7299190206098010112}]
I20260621 08:00:58.933256 2196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.933386 2196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80 [sys.catalog]: This master's current role is: LEADER
I20260621 08:00:58.933612 2224 catalog_manager.cc:1347] Generated new cluster ID: 87261cad6cc84323b209f2b2102dc47d
I20260621 08:00:58.933663 2224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:00:58.934406 2195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c37d120c9d949ea858e096d68b19e80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c37d120c9d949ea858e096d68b19e80" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 43749 } } peers { permanent_uuid: "0bb66cfae9d64cf88698bd08d111fb8a" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 35745 } } peers { permanent_uuid: "611435b40f0c464a8107ed7f65483d25" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 46425 } } }
I20260621 08:00:58.934504 2195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:00:58.934906 2228 mvcc.cc:205] Tried to move back new op lower bound from 7299190206098010112 to 7299190205986926592. Current Snapshot: MvccSnapshot[applied={T|T < 7299190206098010112}]
I20260621 08:00:58.980895 2224 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:00:58.982617 2224 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:00:58.995343 2224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c37d120c9d949ea858e096d68b19e80: Generated new TSK 0
I20260621 08:00:58.996286 2224 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 08:00:59.024569 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.021510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.11.223.189:35745)
0621 08:00:59.021647 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.021651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.023429 (+ 1778us) server_negotiation.cc:407] Connection header received
0621 08:00:59.023638 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.023644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.023693 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.023748 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.024427 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43712: BlockingRecv error: recv got EOF from 127.0.0.1:43712 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260621 08:00:59.026080 2020 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.022278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.11.223.190:43749)
0621 08:00:59.022392 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.022396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.025228 (+ 2832us) server_negotiation.cc:407] Connection header received
0621 08:00:59.025392 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.025395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.025443 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.025492 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.025973 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38872: BlockingRecv error: recv got EOF from 127.0.0.1:38872 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260621 08:00:59.027593 2173 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.022915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51818 (local address 127.11.223.188:46425)
0621 08:00:59.023019 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.023022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.026683 (+ 3661us) server_negotiation.cc:407] Connection header received
0621 08:00:59.026908 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.026911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.026961 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.027010 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.027465 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51818: BlockingRecv error: recv got EOF from 127.0.0.1:51818 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260621 08:00:59.034178 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.032766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.11.223.189:35745)
0621 08:00:59.032864 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.032867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.033291 (+ 424us) server_negotiation.cc:407] Connection header received
0621 08:00:59.033500 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.033503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.033570 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.033631 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.034079 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43728: BlockingRecv error: recv got EOF from 127.0.0.1:43728 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260621 08:00:59.039081 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.037647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.11.223.189:35745)
0621 08:00:59.037745 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.037749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.038260 (+ 511us) server_negotiation.cc:407] Connection header received
0621 08:00:59.038445 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.038448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.038492 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.038536 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.038991 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43736: BlockingRecv error: recv got EOF from 127.0.0.1:43736 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260621 08:00:59.044420 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.043012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.11.223.189:35745)
0621 08:00:59.043103 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.043107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.043562 (+ 455us) server_negotiation.cc:407] Connection header received
0621 08:00:59.043747 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.043750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.043800 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.043852 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.044300 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43738: BlockingRecv error: recv got EOF from 127.0.0.1:43738 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260621 08:00:59.052337 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.050782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.11.223.189:35745)
0621 08:00:59.050906 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.050910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.051397 (+ 487us) server_negotiation.cc:407] Connection header received
0621 08:00:59.051594 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.051597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.051657 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.051709 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.052207 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43744: BlockingRecv error: recv got EOF from 127.0.0.1:43744 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260621 08:00:59.064252 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.062640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.11.223.189:35745)
0621 08:00:59.062742 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.062746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.063301 (+ 555us) server_negotiation.cc:407] Connection header received
0621 08:00:59.063500 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.063503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.063553 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.063614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.064144 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43756: BlockingRecv error: recv got EOF from 127.0.0.1:43756 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260621 08:00:59.084585 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.082834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43768 (local address 127.11.223.189:35745)
0621 08:00:59.082947 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.082952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.083559 (+ 607us) server_negotiation.cc:407] Connection header received
0621 08:00:59.083765 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.083768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.083820 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.083890 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.084451 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43768: BlockingRecv error: recv got EOF from 127.0.0.1:43768 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260621 08:00:59.120434 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.119181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.11.223.189:35745)
0621 08:00:59.119349 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.119353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.119486 (+ 133us) server_negotiation.cc:407] Connection header received
0621 08:00:59.119696 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.119699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.119751 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.119836 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.120287 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43770: BlockingRecv error: recv got EOF from 127.0.0.1:43770 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260621 08:00:59.187567 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.186447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.11.223.189:35745)
0621 08:00:59.186643 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.186647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.186741 (+ 94us) server_negotiation.cc:407] Connection header received
0621 08:00:59.186872 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.186875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.186925 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.187000 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.187432 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43772: BlockingRecv error: recv got EOF from 127.0.0.1:43772 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260621 08:00:59.318745 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.317725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.11.223.189:35745)
0621 08:00:59.317858 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.317861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.318002 (+ 141us) server_negotiation.cc:407] Connection header received
0621 08:00:59.318097 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.318099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.318145 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.318227 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.318637 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43784: BlockingRecv error: recv got EOF from 127.0.0.1:43784 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260621 08:00:59.578192 2095 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:00:59.577086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.11.223.189:35745)
0621 08:00:59.577216 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0621 08:00:59.577219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:00:59.577331 (+ 112us) server_negotiation.cc:407] Connection header received
0621 08:00:59.577431 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:00:59.577434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:00:59.577479 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:00:59.577554 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:00:59.578042 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43800: BlockingRecv error: recv got EOF from 127.0.0.1:43800 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260621 08:00:59.888808 2207 catalog_manager.cc:1259] Loaded cluster ID: 87261cad6cc84323b209f2b2102dc47d
I20260621 08:00:59.888903 2207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: loading cluster ID for follower catalog manager: success
I20260621 08:00:59.890007 2222 catalog_manager.cc:1259] Loaded cluster ID: 87261cad6cc84323b209f2b2102dc47d
I20260621 08:00:59.890085 2222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: loading cluster ID for follower catalog manager: success
I20260621 08:00:59.891011 2207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: acquiring CA information for follower catalog manager: success
I20260621 08:00:59.891675 2222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: acquiring CA information for follower catalog manager: success
I20260621 08:00:59.892215 2207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 611435b40f0c464a8107ed7f65483d25: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 08:00:59.892760 2222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bb66cfae9d64cf88698bd08d111fb8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 08:01:00.096278 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:00.092244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.11.223.189:35745)
0621 08:01:00.092506 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:00.092511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:00.092679 (+ 168us) server_negotiation.cc:407] Connection header received
0621 08:01:00.092853 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:00.092858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:00.092998 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:00.093126 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:00.093390 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.093911 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.094781 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.095035 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.095146 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:00.095453 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:00.095887 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:00.096027 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:00.096066 (+ 39us) server_negotiation.cc:299] Negotiation successful
0621 08:01:00.096119 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
I20260621 08:01:00.102325 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:00.099010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.11.223.188:46425)
0621 08:01:00.099203 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:00.099207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:00.099219 (+ 12us) server_negotiation.cc:407] Connection header received
0621 08:01:00.099351 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:00.099353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:00.099460 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:00.099555 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:00.099724 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.100215 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.100993 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.101197 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.101269 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:00.101515 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:00.101942 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:00.102034 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:00.102080 (+ 46us) server_negotiation.cc:299] Negotiation successful
0621 08:01:00.102125 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":60,"threads_started":1}
I20260621 08:01:00.109102 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:00.105029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43818 (local address 127.11.223.189:35745)
0621 08:01:00.105132 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:00.105135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:00.105548 (+ 413us) server_negotiation.cc:407] Connection header received
0621 08:01:00.105723 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:00.105726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:00.105790 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:00.105875 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:00.106106 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.106931 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.107889 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.108196 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.108288 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:00.108584 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:00.108662 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:00.108709 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:00.108898 (+ 189us) server_negotiation.cc:299] Negotiation successful
0621 08:01:00.108950 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260621 08:01:00.109330 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:00.105108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.11.223.188:46425)
0621 08:01:00.105222 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:00.105226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:00.105603 (+ 377us) server_negotiation.cc:407] Connection header received
0621 08:01:00.105794 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:00.105797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:00.105874 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:00.105934 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:00.106264 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.107010 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.108142 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.108451 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.108548 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:00.108851 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:00.108919 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:00.108984 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:00.109157 (+ 173us) server_negotiation.cc:299] Negotiation successful
0621 08:01:00.109212 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260621 08:01:00.109911 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:00.105276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.11.223.190:43749)
0621 08:01:00.105516 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:00.105521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:00.105540 (+ 19us) server_negotiation.cc:407] Connection header received
0621 08:01:00.105642 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:00.105645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:00.105839 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:00.105967 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:00.106290 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.106822 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.107998 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:00.108395 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:00.108510 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:00.108806 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:00.109312 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:00.109426 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:00.109585 (+ 159us) server_negotiation.cc:299] Negotiation successful
0621 08:01:00.109771 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":64,"threads_started":1}
I20260621 08:01:00.115911 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 1950
I20260621 08:01:00.121977 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 2025
I20260621 08:01:00.127660 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 2100
2026-06-21T08:01:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3098 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 21 08:01:00 dist-test-slave-0pn3 krb5kdc[2317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 21 08:01:00 dist-test-slave-0pn3 krb5kdc[2317](info): set up 2 sockets
Jun 21 08:01:00 dist-test-slave-0pn3 krb5kdc[2317](info): commencing operation
krb5kdc: starting...
W20260621 08:01:01.168998 12158 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/test-user.keytab.
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-21T08:01:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-21T08:01:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.223.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
Entry for principal kudu/127.11.223.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab.
I20260621 08:01:01.287341 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:35363
--webserver_interface=127.11.223.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:35363,127.11.223.189:46057,127.11.223.188:36367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260621 08:01:01.397840 2330 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 08:01:01.398144 2330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:01:01.398196 2330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:01:01.398275 2330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:01:01.402205 2330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:01:01.402279 2330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:01:01.402316 2330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:01:01.402367 2330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:01:01.402403 2330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:01:01.407619 2330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:35363,127.11.223.189:46057,127.11.223.188:36367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.190.keytab
--principal=kudu/127.11.223.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.190:35363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.223.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:01:01.408704 2330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:01:01.409709 2330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:01:01.417838 2336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:01:01.417891 2330 server_base.cc:1034] running on GCE node
W20260621 08:01:01.417861 2338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:01:01.418092 2335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:01:01.418609 2330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:01:01.419608 2330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:01:01.420754 2330 hybrid_clock.cc:648] HybridClock initialized: now 1782028861420721 us; error 46 us; skew 500 ppm
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:01:01.423254 2330 init.cc:377] Logged in from keytab as kudu/127.11.223.190@KRBTEST.COM (short username kudu)
I20260621 08:01:01.424065 2330 webserver.cc:460] Webserver started at http://127.11.223.190:35505/ using document root <none> and password file <none>
I20260621 08:01:01.424651 2330 fs_manager.cc:359] Metadata directory not provided
I20260621 08:01:01.424714 2330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:01:01.424881 2330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:01:01.426762 2330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data/instance:
uuid: "2abebb965d2a4bb1a54dc05c184911c8"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "ee8dcbb244175527ab4530a7cb9577c2"
server_key_iv: "2718c7e1702d3b0a7dedfec90b73ad18"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.427268 2330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal/instance:
uuid: "2abebb965d2a4bb1a54dc05c184911c8"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "ee8dcbb244175527ab4530a7cb9577c2"
server_key_iv: "2718c7e1702d3b0a7dedfec90b73ad18"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.430826 2330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260621 08:01:01.433125 2345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:01:01.434304 2330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:01:01.434432 2330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal
uuid: "2abebb965d2a4bb1a54dc05c184911c8"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "ee8dcbb244175527ab4530a7cb9577c2"
server_key_iv: "2718c7e1702d3b0a7dedfec90b73ad18"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.434522 2330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:01:01.458019 2330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:01:01.461302 2330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:01:01.461511 2330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:01:01.468180 2330 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.190:35363
I20260621 08:01:01.468514 2397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.190:35363 every 8 connection(s)
I20260621 08:01:01.469184 2330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/data/info.pb
I20260621 08:01:01.471856 2398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:01:01.473670 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 2330
I20260621 08:01:01.473793 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-0/wal/instance
I20260621 08:01:01.474077 12158 external_mini_cluster.cc:1428] Setting key c4a7e1986e3d7f0d816f1a8de1bf5de8
I20260621 08:01:01.476657 2398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.223.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.189@KRBTEST.COM" created.
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:01:01.492189 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.478830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.11.223.190:35363)
0621 08:01:01.479394 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.479411 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.479463 (+ 52us) server_negotiation.cc:407] Connection header received
0621 08:01:01.480484 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.480490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.480756 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.480987 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.481766 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.483103 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.484108 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.484604 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.484720 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.487395 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.487401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.487410 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.487429 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.489126 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.489581 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.489592 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.489594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.489694 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.489992 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.489997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.489999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.490363 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.490497 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.490856 (+ 359us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.491102 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":443,"spinlock_wait_cycles":2816,"thread_start_us":168,"threads_started":1}
I20260621 08:01:01.492192 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.478815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:35363 (local address 127.0.0.1:43684)
0621 08:01:01.479292 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.479344 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.480142 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.480970 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.480974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.481299 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.481620 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.481633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.483222 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.483225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.484001 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.484009 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.484746 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.484749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.484835 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.485587 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.485608 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.487244 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.489232 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.489242 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.489252 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.489476 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.489788 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.489791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.489793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.489902 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.490480 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.490485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.490623 (+ 138us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.490791 (+ 168us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.491152 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":135,"thread_start_us":148,"threads_started":1}
I20260621 08:01:01.494607 2398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:01:01.495362 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.495111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50928)
0621 08:01:01.495221 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.495289 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260621 08:01:01.495543 2349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
I20260621 08:01:01.496543 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.496235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50942)
0621 08:01:01.496397 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.496470 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260621 08:01:01.496758 2398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:46057: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab.
Entry for principal kudu/127.11.223.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab.
I20260621 08:01:01.499429 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:46057
--webserver_interface=127.11.223.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab
--principal=kudu/127.11.223.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:35363,127.11.223.189:46057,127.11.223.188:36367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:01:01.546080 2398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } attempt: 1
I20260621 08:01:01.547030 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.546725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50944)
0621 08:01:01.546882 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.546949 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260621 08:01:01.547788 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.547539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50956)
0621 08:01:01.547649 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.547719 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260621 08:01:01.547921 2398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:46057: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
W20260621 08:01:01.614657 2405 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 08:01:01.614977 2405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:01:01.615034 2405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:01:01.615121 2405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260621 08:01:01.616075 2398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } attempt: 2
I20260621 08:01:01.616911 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.616625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50968)
0621 08:01:01.616771 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.616836 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260621 08:01:01.617574 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.617364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50976)
0621 08:01:01.617468 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.617516 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260621 08:01:01.617723 2398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.189:46057: Network error: Client connection negotiation failed: client connection to 127.11.223.189:46057: connect: Connection refused (error 111)
W20260621 08:01:01.619516 2405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:01:01.619617 2405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:01:01.619660 2405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:01:01.619702 2405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:01:01.619737 2405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:01:01.625239 2405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:35363,127.11.223.189:46057,127.11.223.188:36367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.189.keytab
--principal=kudu/127.11.223.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.189:46057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.223.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:01:01.626327 2405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:01:01.627391 2405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:01:01.635910 2411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:01:01.635932 2413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:01:01.635913 2410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:01:01.636168 2405 server_base.cc:1034] running on GCE node
I20260621 08:01:01.636828 2405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:01:01.637960 2405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:01:01.639178 2405 hybrid_clock.cc:648] HybridClock initialized: now 1782028861639136 us; error 59 us; skew 500 ppm
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:01:01.642004 2405 init.cc:377] Logged in from keytab as kudu/127.11.223.189@KRBTEST.COM (short username kudu)
I20260621 08:01:01.642865 2405 webserver.cc:460] Webserver started at http://127.11.223.189:43809/ using document root <none> and password file <none>
I20260621 08:01:01.643379 2405 fs_manager.cc:359] Metadata directory not provided
I20260621 08:01:01.643436 2405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:01:01.643621 2405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:01:01.645841 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data/instance:
uuid: "b51ae552025f4af28b0c44f46e0adca4"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "04c1a2e8d908bb3829dd2e1e2b53778b"
server_key_iv: "5583f4844a143ae92343604e00ec089c"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.646361 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal/instance:
uuid: "b51ae552025f4af28b0c44f46e0adca4"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "04c1a2e8d908bb3829dd2e1e2b53778b"
server_key_iv: "5583f4844a143ae92343604e00ec089c"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.650125 2405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260621 08:01:01.652562 2420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:01:01.653842 2405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260621 08:01:01.653970 2405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal
uuid: "b51ae552025f4af28b0c44f46e0adca4"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "04c1a2e8d908bb3829dd2e1e2b53778b"
server_key_iv: "5583f4844a143ae92343604e00ec089c"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.654079 2405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:01:01.676630 2405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:01:01.680109 2405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:01:01.680287 2405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:01:01.687418 2405 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.189:46057
I20260621 08:01:01.687733 2472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.189:46057 every 8 connection(s)
I20260621 08:01:01.688495 2405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/data/info.pb
I20260621 08:01:01.691344 2473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:01:01.695257 2473 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:01:01.696162 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 2405
I20260621 08:01:01.696259 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-1/wal/instance
I20260621 08:01:01.696575 12158 external_mini_cluster.cc:1428] Setting key 2eeb88c2f322911203f7043401795da1
WARNING: no policy specified for kudu/127.11.223.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.223.188@KRBTEST.COM" created.
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:01:01.708879 2398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } attempt: 3
I20260621 08:01:01.713637 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.697351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.11.223.190:35363)
0621 08:01:01.697501 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.697505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.698665 (+ 1160us) server_negotiation.cc:407] Connection header received
0621 08:01:01.699989 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.699995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.700065 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.700167 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.701439 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.702379 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.703512 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.703933 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.704066 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.708453 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.708456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.708458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.708485 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.710539 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.711151 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.711155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.711157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.711253 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.711645 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.711651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.711656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.711919 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.712029 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.713310 (+ 1281us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.713446 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260621 08:01:01.715066 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.697847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:35363 (local address 127.0.0.1:43688)
0621 08:01:01.698464 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.698530 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.699762 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.700368 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.700376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.700737 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.701282 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.701296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.702507 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.702510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.703376 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.703385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.704085 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.704087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.704238 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.704906 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.704923 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.708231 (+ 3308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.710649 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.710656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.710677 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.711043 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.711361 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.711365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.711367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.711541 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.712031 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.712038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.712391 (+ 353us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.713771 (+ 1380us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.714145 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":177,"threads_started":1}
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 08:01:01.716988 2473 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:01:01.719939 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.709352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50986)
0621 08:01:01.709520 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.709535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.709624 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.710178 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.710182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.710493 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.710666 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.710683 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.711912 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.711916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.712755 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.712764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.713657 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.713663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.713805 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.714342 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.714362 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.715920 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.718285 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.718289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.718291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.718481 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.718785 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.718793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.718795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.718867 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.719341 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.719345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.719445 (+ 100us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.719547 (+ 102us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.719694 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260621 08:01:01.719964 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.709459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.11.223.189:46057)
0621 08:01:01.709796 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.709805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.709829 (+ 24us) server_negotiation.cc:407] Connection header received
0621 08:01:01.709865 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.709870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.710039 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.710153 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.710798 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.711801 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.712886 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.713520 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.713625 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.716054 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.716069 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.716098 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.716125 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.718183 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.718596 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.718601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.718603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.718688 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.718954 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.718959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.718960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.719243 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.719378 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.719659 (+ 281us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.719797 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":146,"threads_started":1}
I20260621 08:01:01.722339 2398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:01:01.723210 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.722918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45312)
0621 08:01:01.723096 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.723146 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.223.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab.
Entry for principal kudu/127.11.223.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab.
I20260621 08:01:01.724099 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.723760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45328)
0621 08:01:01.723943 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.724002 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260621 08:01:01.724253 2398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:36367: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
I20260621 08:01:01.725455 12158 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
/tmp/dist-test-taskKM8XKG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:36367
--webserver_interface=127.11.223.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab
--principal=kudu/127.11.223.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.223.148:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.223.190:35363,127.11.223.189:46057,127.11.223.188:36367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260621 08:01:01.727488 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.717548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50996)
0621 08:01:01.717665 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.717681 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.717757 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.718217 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.718223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.718379 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.718468 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.718518 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.719449 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.719455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.720091 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.720103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.720687 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.720690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.720834 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.722137 (+ 1303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.722148 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.723608 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.725704 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.725709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.725714 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.725956 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.726274 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.726277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.726279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.726397 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.726854 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.726856 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.726986 (+ 130us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.727111 (+ 125us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.727275 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260621 08:01:01.727494 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.717689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.11.223.189:46057)
0621 08:01:01.717977 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.717984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.717996 (+ 12us) server_negotiation.cc:407] Connection header received
0621 08:01:01.718052 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.718055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.718098 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.718187 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.718660 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.719306 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.720223 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.720578 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.720688 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.723746 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.723749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.723751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.723790 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.725582 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.726057 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.726063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.726068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.726146 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.726484 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.726489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.726494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.726757 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.726865 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.727172 (+ 307us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.727363 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":235,"thread_start_us":119,"threads_started":1}
I20260621 08:01:01.728159 2473 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:01:01.728906 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.728591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45336)
0621 08:01:01.728739 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.728827 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260621 08:01:01.729127 2421 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
I20260621 08:01:01.730057 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.729802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45342)
0621 08:01:01.729932 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.729990 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260621 08:01:01.730257 2473 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:36367: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
I20260621 08:01:01.755425 2398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } attempt: 1
I20260621 08:01:01.756464 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.756103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45352)
0621 08:01:01.756263 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.756380 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260621 08:01:01.757182 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.756987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45362)
0621 08:01:01.757086 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.757130 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260621 08:01:01.757383 2398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:36367: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
I20260621 08:01:01.779551 2473 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } attempt: 1
I20260621 08:01:01.780525 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.780140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45372)
0621 08:01:01.780344 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.780422 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260621 08:01:01.781277 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.781050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45386)
0621 08:01:01.781161 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.781214 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260621 08:01:01.781430 2473 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:36367: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
I20260621 08:01:01.832535 2398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } attempt: 2
I20260621 08:01:01.833376 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.833074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45390)
0621 08:01:01.833222 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.833291 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260621 08:01:01.834218 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.833984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45404)
0621 08:01:01.834092 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.834141 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260621 08:01:01.834403 2398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:36367: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
W20260621 08:01:01.840114 2480 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260621 08:01:01.840502 2480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260621 08:01:01.840550 2480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260621 08:01:01.840632 2480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260621 08:01:01.844897 2480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260621 08:01:01.844956 2480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260621 08:01:01.844995 2480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260621 08:01:01.845031 2480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260621 08:01:01.845075 2480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260621 08:01:01.849601 2473 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } attempt: 2
I20260621 08:01:01.850499 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.850129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45412)
0621 08:01:01.850306 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.850387 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260621 08:01:01.851065 2480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.223.148:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.223.190:35363,127.11.223.189:46057,127.11.223.188:36367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/krb5kdc/kudu_127.11.223.188.keytab
--principal=kudu/127.11.223.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.223.188:36367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.223.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 21 Jun 2026 07:00:36 UTC on ccycloud.quasar-apkvod.root.comops.site
build id 4457933
I20260621 08:01:01.851222 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.851000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45418)
0621 08:01:01.851114 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.851163 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260621 08:01:01.851341 2473 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.223.188:36367: Network error: Client connection negotiation failed: client connection to 127.11.223.188:36367: connect: Connection refused (error 111)
I20260621 08:01:01.852238 2480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260621 08:01:01.853344 2480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260621 08:01:01.861718 2489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:01:01.861713 2486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260621 08:01:01.861713 2487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260621 08:01:01.862082 2480 server_base.cc:1034] running on GCE node
I20260621 08:01:01.862645 2480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260621 08:01:01.863781 2480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260621 08:01:01.864961 2480 hybrid_clock.cc:648] HybridClock initialized: now 1782028861864946 us; error 32 us; skew 500 ppm
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260621 08:01:01.867671 2480 init.cc:377] Logged in from keytab as kudu/127.11.223.188@KRBTEST.COM (short username kudu)
I20260621 08:01:01.868575 2480 webserver.cc:460] Webserver started at http://127.11.223.188:37803/ using document root <none> and password file <none>
I20260621 08:01:01.869153 2480 fs_manager.cc:359] Metadata directory not provided
I20260621 08:01:01.869217 2480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260621 08:01:01.869441 2480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260621 08:01:01.871462 2480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data/instance:
uuid: "3f7e4d624437485c8478ef3eb20a919c"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "a619974e279c49b9f03dfec6a53567cb"
server_key_iv: "5822862f8305f7dd834cb93c997152c8"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.872025 2480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal/instance:
uuid: "3f7e4d624437485c8478ef3eb20a919c"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "a619974e279c49b9f03dfec6a53567cb"
server_key_iv: "5822862f8305f7dd834cb93c997152c8"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.875816 2480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260621 08:01:01.878257 2496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260621 08:01:01.879532 2480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260621 08:01:01.879670 2480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data,/tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal
uuid: "3f7e4d624437485c8478ef3eb20a919c"
format_stamp: "Formatted at 2026-06-21 08:01:01 on dist-test-slave-0pn3"
server_key: "a619974e279c49b9f03dfec6a53567cb"
server_key_iv: "5822862f8305f7dd834cb93c997152c8"
server_key_version: "encryptionkey@0"
I20260621 08:01:01.879799 2480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260621 08:01:01.890002 2480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260621 08:01:01.893381 2480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260621 08:01:01.893594 2480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260621 08:01:01.900645 2480 rpc_server.cc:307] RPC server started. Bound to: 127.11.223.188:36367
I20260621 08:01:01.900934 2548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.223.188:36367 every 8 connection(s)
I20260621 08:01:01.901695 2480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/data/info.pb
I20260621 08:01:01.902784 12158 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu as pid 2480
I20260621 08:01:01.902925 12158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKM8XKG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1782028587408853-12158-0/minicluster-data/master-2/wal/instance
I20260621 08:01:01.903224 12158 external_mini_cluster.cc:1428] Setting key 8c33bd640db66393da17d4ec8f1f4de1
I20260621 08:01:01.905125 2549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260621 08:01:01.908977 2549 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
I20260621 08:01:01.917858 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.904682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.11.223.188:36367)
0621 08:01:01.905120 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.905136 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.905172 (+ 36us) server_negotiation.cc:407] Connection header received
0621 08:01:01.905887 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.905904 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.906252 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.906612 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.907461 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.908539 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.909434 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.909906 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.910083 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.912872 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.912878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.912890 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.912923 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.914762 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.915202 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.915206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.915211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.915378 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.915664 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.915668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.915670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.916137 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.916367 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.916677 (+ 310us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.917072 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":134,"threads_started":1}
I20260621 08:01:01.919865 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.910762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.11.223.190:35363)
0621 08:01:01.910940 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.910944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.911324 (+ 380us) server_negotiation.cc:407] Connection header received
0621 08:01:01.911503 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.911506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.911561 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.911650 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.912296 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.912895 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.913761 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.914071 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.914193 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.916402 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.916406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.916409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.916436 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.917881 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.918526 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.918531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.918532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.918631 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.918924 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.918927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.918928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.919118 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.919200 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.919566 (+ 366us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.919718 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260621 08:01:01.919870 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.910733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.190:35363 (local address 127.0.0.1:43694)
0621 08:01:01.911178 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.911226 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.911389 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.911708 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.911712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.912045 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.912169 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.912174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.913014 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.913017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.913642 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.913650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.914189 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.914192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.914281 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.914770 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.914782 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.916239 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.918183 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.918193 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.918199 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.918427 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.918728 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.918731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.918733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.918829 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.919214 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.919219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.919343 (+ 124us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.919486 (+ 143us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.919680 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":181,"threads_started":1}
I20260621 08:01:01.921386 2549 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } has no permanent_uuid. Determining permanent_uuid...
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.190@KRBTEST.COM
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
I20260621 08:01:01.929330 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.920261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.11.223.190:35363)
0621 08:01:01.920422 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.920426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.920465 (+ 39us) server_negotiation.cc:407] Connection header received
0621 08:01:01.920643 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.920646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.920702 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.920782 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.921478 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.922207 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.922990 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.923213 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.923326 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.926070 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.926076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.926081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.926128 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.927709 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.928120 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.928125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.928127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.928212 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.928522 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.928526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.928527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.928706 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.928967 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.929049 (+ 82us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.929159 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260621 08:01:01.929355 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.921883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.189:46057 (local address 127.0.0.1:50998)
0621 08:01:01.921991 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.922005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.922075 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.922355 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.922359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.922523 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.922611 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.922616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.923354 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.923358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.923849 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.923856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.924392 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.924395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.924492 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.924975 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.924993 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.926320 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.927883 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.927888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.927890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.928083 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.928379 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.928383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.928385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.928469 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.928837 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.928840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.928930 (+ 90us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.929036 (+ 106us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.929168 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260621 08:01:01.929806 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.922010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.11.223.189:46057)
0621 08:01:01.922150 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.922154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.922168 (+ 14us) server_negotiation.cc:407] Connection header received
0621 08:01:01.922213 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.922215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.922255 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.922338 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.922725 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.923247 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.923961 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.924241 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.924392 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.926452 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.926455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.926457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.926481 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.927764 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.928179 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.928184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.928186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.928265 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.928557 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.928562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.928563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.928743 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.928854 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.929098 (+ 244us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.929217 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 08:01:01.930431 2549 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } has no permanent_uuid. Determining permanent_uuid...
I20260621 08:01:01.933583 2398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } attempt: 3
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.188@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.223.189@KRBTEST.COM
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.190@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
I20260621 08:01:01.939864 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.930951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.11.223.188:36367)
0621 08:01:01.931056 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.931061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.931076 (+ 15us) server_negotiation.cc:407] Connection header received
0621 08:01:01.931167 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.931170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.931211 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.931289 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.931640 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.932693 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.933363 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.933742 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.933852 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.936392 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.936397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.936404 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.936429 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.938084 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.938548 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.938553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.938554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.938632 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.938915 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.938918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.938919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.939081 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.939194 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.939486 (+ 292us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.939651 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":42}
I20260621 08:01:01.940212 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.930900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45446)
0621 08:01:01.930988 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.931001 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.931067 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.931301 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.931304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.931464 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.931531 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.931536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.932782 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.932785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.933250 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.933257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.933967 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.933970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.934042 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.934478 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.934496 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.936233 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.938195 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.938200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.938203 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.938439 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.938736 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.938739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.938741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.938827 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.939233 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.939235 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.939309 (+ 74us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.939431 (+ 122us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.939572 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260621 08:01:01.941085 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.930756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.11.223.189:46057)
0621 08:01:01.930864 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.930868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.930967 (+ 99us) server_negotiation.cc:407] Connection header received
0621 08:01:01.931147 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.931150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.931201 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.931289 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.932019 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.932615 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.933384 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.933614 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.933749 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.936566 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.936569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.936572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.936596 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.938427 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.939391 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.939395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.939397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.939481 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.939777 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.939781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.939782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.940090 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.940289 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.940813 (+ 524us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.940939 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260621 08:01:01.942490 2473 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } attempt: 3
I20260621 08:01:01.943881 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.933961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45454)
0621 08:01:01.934445 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.934460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.934537 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.935299 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.935301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.935443 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.935515 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.935520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.936425 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.936429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.937266 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.937275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.937926 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.937930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.938009 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.938576 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.938590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.940007 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.942249 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.942254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.942256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.942503 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.942841 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.942844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.942845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.942938 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.943388 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.943391 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.943486 (+ 95us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.943571 (+ 85us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.943708 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404}
I20260621 08:01:01.944017 2549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: Bootstrap starting.
I20260621 08:01:01.944533 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.934132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.11.223.188:36367)
0621 08:01:01.935082 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.935093 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.935105 (+ 12us) server_negotiation.cc:407] Connection header received
0621 08:01:01.935137 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.935141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.935181 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.935269 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.935670 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.936297 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.937392 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.937805 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.937913 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.940132 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.940137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.940140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.940162 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.942137 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.942618 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.942623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.942627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.942719 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.943062 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.943065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.943067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.943275 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.944142 (+ 867us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.944230 (+ 88us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.944374 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"thread_start_us":153,"threads_started":1}
I20260621 08:01:01.945089 12158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260621 08:01:01.946902 2549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: Neither blocks nor log segments found. Creating new log.
I20260621 08:01:01.947490 2398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: Bootstrap starting.
I20260621 08:01:01.946313 2372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2abebb965d2a4bb1a54dc05c184911c8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43704
Jun 21 08:01:01 dist-test-slave-0pn3 krb5kdc[2317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1782028861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.223.189@KRBTEST.COM for kudu/127.11.223.188@KRBTEST.COM
I20260621 08:01:01.947829 2549 log.cc:824] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: Log is configured to *not* fsync() on all Append() calls
I20260621 08:01:01.950170 2398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: Neither blocks nor log segments found. Creating new log.
I20260621 08:01:01.950253 2549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: No bootstrap required, opened a new log
I20260621 08:01:01.951045 2398 log.cc:824] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: Log is configured to *not* fsync() on all Append() calls
I20260621 08:01:01.951740 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.942842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.223.188:36367 (local address 127.0.0.1:45460)
0621 08:01:01.943064 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0621 08:01:01.943078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0621 08:01:01.943149 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0621 08:01:01.943610 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0621 08:01:01.943613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0621 08:01:01.943775 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0621 08:01:01.943868 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.943875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.944811 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.944815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.945577 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0621 08:01:01.945587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.946198 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.946202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0621 08:01:01.946280 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.946746 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0621 08:01:01.946765 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0621 08:01:01.948170 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0621 08:01:01.950207 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.950212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.950214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.950439 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.950726 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0621 08:01:01.950729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0621 08:01:01.950731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0621 08:01:01.950810 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0621 08:01:01.951249 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0621 08:01:01.951252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0621 08:01:01.951343 (+ 91us) client_negotiation.cc:769] Sending connection context
0621 08:01:01.951426 (+ 83us) client_negotiation.cc:240] Negotiation successful
0621 08:01:01.951571 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260621 08:01:01.952389 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.943278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45460 (local address 127.11.223.188:36367)
0621 08:01:01.943371 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.943374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.943386 (+ 12us) server_negotiation.cc:407] Connection header received
0621 08:01:01.943414 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.943417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.943451 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.943514 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.943990 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.944688 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.945715 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.946001 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.946086 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.948351 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.948354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.948356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.948377 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.950104 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.950556 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.950562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.950565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.950642 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.950920 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.950925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.950927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.951124 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.952020 (+ 896us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.952090 (+ 70us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.952211 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260621 08:01:01.953105 2398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: No bootstrap required, opened a new log
I20260621 08:01:01.954352 2549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.954593 2549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f7e4d624437485c8478ef3eb20a919c, State: Initialized, Role: FOLLOWER
I20260621 08:01:01.954947 2549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.955161 2473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: Bootstrap starting.
I20260621 08:01:01.955518 2557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:01.955739 2557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:01:01.956053 2398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.956283 2398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2abebb965d2a4bb1a54dc05c184911c8, State: Initialized, Role: FOLLOWER
I20260621 08:01:01.956364 2549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:01:01.956664 2398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.957252 2558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:01.957456 2558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:01:01.957880 2473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: Neither blocks nor log segments found. Creating new log.
I20260621 08:01:01.958038 2398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:01:01.958143 2372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260621 08:01:01.958256 2372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:01:01.959527 2372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.960558 2372 leader_election.cc:288] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [CANDIDATE]: Term 1 election: Requested vote from peers b51ae552025f4af28b0c44f46e0adca4 (127.11.223.189:46057), 3f7e4d624437485c8478ef3eb20a919c (127.11.223.188:36367)
I20260621 08:01:01.961357 2523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2abebb965d2a4bb1a54dc05c184911c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3f7e4d624437485c8478ef3eb20a919c"
I20260621 08:01:01.961589 2523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [term 0 FOLLOWER]: Advancing to term 1
W20260621 08:01:01.961830 2569 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:01:01.961896 2569 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:01:01.963001 2523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2abebb965d2a4bb1a54dc05c184911c8 in term 1.
W20260621 08:01:01.963029 2351 tablet.cc:2369] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260621 08:01:01.962859 2473 log.cc:824] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: Log is configured to *not* fsync() on all Append() calls
I20260621 08:01:01.963869 2346 leader_election.cc:302] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2abebb965d2a4bb1a54dc05c184911c8, 3f7e4d624437485c8478ef3eb20a919c; no voters:
I20260621 08:01:01.960166 2480 master_runner.cc:186] Error getting master registration for 127.11.223.190:35363: OK, instance_id { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" instance_seqno: 1782028861466427 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 08:01:01.964072 2480 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 08:01:01.966300 2473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: No bootstrap required, opened a new log
I20260621 08:01:01.965593 2558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260621 08:01:01.966701 2558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [term 1 LEADER]: Becoming Leader. State: Replica: 2abebb965d2a4bb1a54dc05c184911c8, State: Running, Role: LEADER
I20260621 08:01:01.967290 2558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.966770 2330 master_runner.cc:186] Error getting master registration for 127.11.223.189:46057: OK, instance_id { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" instance_seqno: 1782028861685381 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260621 08:01:01.970145 2330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260621 08:01:01.970628 2558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2abebb965d2a4bb1a54dc05c184911c8. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:01.970902 2558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: This master's current role is: LEADER
I20260621 08:01:01.970844 2473 raft_consensus.cc:348] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.971222 2473 raft_consensus.cc:729] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b51ae552025f4af28b0c44f46e0adca4, State: Initialized, Role: FOLLOWER
I20260621 08:01:01.971604 2473 consensus_queue.cc:260] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } }
I20260621 08:01:01.973196 2473 sys_catalog.cc:564] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260621 08:01:01.973249 2447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2abebb965d2a4bb1a54dc05c184911c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b51ae552025f4af28b0c44f46e0adca4"
I20260621 08:01:01.973469 2447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [term 0 FOLLOWER]: Advancing to term 1
I20260621 08:01:01.975229 2447 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2abebb965d2a4bb1a54dc05c184911c8 in term 1.
I20260621 08:01:01.976579 2593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:01.976709 2593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: This master's current role is: FOLLOWER
W20260621 08:01:01.980232 2604 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:01:01.980329 2604 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:01:01.985088 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.965495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.11.223.190:35363)
0621 08:01:01.970298 (+ 4803us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.970304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.971618 (+ 1314us) server_negotiation.cc:407] Connection header received
0621 08:01:01.971842 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.971845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.971899 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.971955 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.973118 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.973914 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.976643 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.976980 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.977053 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.980795 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.980800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.980805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.980834 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.982740 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.983381 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.983388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.983391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.983481 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.983835 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.983840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.983844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.984111 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.984245 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.984772 (+ 527us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.984901 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4715}
I20260621 08:01:01.986953 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.970966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.11.223.188:36367)
0621 08:01:01.971095 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.971100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.975368 (+ 4268us) server_negotiation.cc:407] Connection header received
0621 08:01:01.975567 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.975570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.975629 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.975692 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.978916 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.979612 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.980705 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.981049 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.981162 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.983087 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.983091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.983094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.983124 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.984896 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.985574 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.985578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.985579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.985665 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.985992 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.985995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.985997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.986223 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.986330 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.986664 (+ 334us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.986780 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260621 08:01:01.987528 2581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260621 08:01:01.987637 2581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260621 08:01:01.988144 2591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260621 08:01:01.989024 2591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260621 08:01:01.994410 2523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [term 1 FOLLOWER]: Refusing update from remote peer 2abebb965d2a4bb1a54dc05c184911c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:01:01.995199 2558 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:01:01.996691 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:01.964973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.11.223.189:46057)
0621 08:01:01.965115 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:01.965120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:01.971636 (+ 6516us) server_negotiation.cc:407] Connection header received
0621 08:01:01.972060 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:01.972066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:01.972118 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:01.972170 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:01.976855 (+ 4685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.977616 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.990729 (+ 13113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:01.991066 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:01.991147 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:01.992994 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0621 08:01:01.992998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0621 08:01:01.993001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0621 08:01:01.993027 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0621 08:01:01.994688 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.995205 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.995209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.995214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.995302 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0621 08:01:01.995604 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0621 08:01:01.995607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0621 08:01:01.995609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0621 08:01:01.995866 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0621 08:01:01.995974 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:01.996397 (+ 423us) server_negotiation.cc:299] Negotiation successful
0621 08:01:01.996522 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260621 08:01:01.999517 2557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2abebb965d2a4bb1a54dc05c184911c8. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:01.999640 2557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:01:02.002025 2447 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [term 1 FOLLOWER]: Refusing update from remote peer 2abebb965d2a4bb1a54dc05c184911c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260621 08:01:02.002635 2558 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260621 08:01:02.006858 2593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2abebb965d2a4bb1a54dc05c184911c8. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:02.006983 2593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:01:02.013569 2588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:02.015179 2588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: This master's current role is: LEADER
I20260621 08:01:02.015368 2607 mvcc.cc:205] Tried to move back new op lower bound from 7299190218720288768 to 7299190218623897600. Current Snapshot: MvccSnapshot[applied={T|T < 7299190218720288768}]
I20260621 08:01:02.016469 2557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:02.016604 2557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:01:02.019902 2588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:02.020040 2588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8 [sys.catalog]: This master's current role is: LEADER
I20260621 08:01:02.021275 2593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2abebb965d2a4bb1a54dc05c184911c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2abebb965d2a4bb1a54dc05c184911c8" member_type: VOTER last_known_addr { host: "127.11.223.190" port: 35363 } } peers { permanent_uuid: "b51ae552025f4af28b0c44f46e0adca4" member_type: VOTER last_known_addr { host: "127.11.223.189" port: 46057 } } peers { permanent_uuid: "3f7e4d624437485c8478ef3eb20a919c" member_type: VOTER last_known_addr { host: "127.11.223.188" port: 36367 } } }
I20260621 08:01:02.021402 2593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4 [sys.catalog]: This master's current role is: FOLLOWER
I20260621 08:01:02.021567 2591 catalog_manager.cc:1347] Generated new cluster ID: 9f92d49fec924324b92a8f733a5eb808
I20260621 08:01:02.021639 2591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260621 08:01:02.031666 2610 mvcc.cc:205] Tried to move back new op lower bound from 7299190218720288768 to 7299190218623897600. Current Snapshot: MvccSnapshot[applied={T|T < 7299190218720288768 or (T in {7299190218720288768})}]
I20260621 08:01:02.069278 2591 catalog_manager.cc:1370] Generated new certificate authority record
I20260621 08:01:02.070611 2591 catalog_manager.cc:1504] Loading token signing keys...
I20260621 08:01:02.085000 2591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2abebb965d2a4bb1a54dc05c184911c8: Generated new TSK 0
I20260621 08:01:02.085990 2591 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260621 08:01:02.119001 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.117020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.11.223.189:46057)
0621 08:01:02.117166 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.117171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.117866 (+ 695us) server_negotiation.cc:407] Connection header received
0621 08:01:02.118074 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.118077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.118128 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.118195 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.118844 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51022: BlockingRecv error: recv got EOF from 127.0.0.1:51022 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260621 08:01:02.121359 2402 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.119699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.11.223.190:35363)
0621 08:01:02.119814 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.119819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.120442 (+ 623us) server_negotiation.cc:407] Connection header received
0621 08:01:02.120612 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.120615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.120664 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.120716 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.121235 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43728: BlockingRecv error: recv got EOF from 127.0.0.1:43728 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260621 08:01:02.122489 2552 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.120185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.11.223.188:36367)
0621 08:01:02.120314 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.120318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.121744 (+ 1426us) server_negotiation.cc:407] Connection header received
0621 08:01:02.121859 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.121862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.121912 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.121994 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.122370 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45476: BlockingRecv error: recv got EOF from 127.0.0.1:45476 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260621 08:01:02.126616 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.125090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.11.223.189:46057)
0621 08:01:02.125206 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.125210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.125636 (+ 426us) server_negotiation.cc:407] Connection header received
0621 08:01:02.125846 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.125849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.125920 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.125979 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.126512 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51032: BlockingRecv error: recv got EOF from 127.0.0.1:51032 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260621 08:01:02.132014 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.130321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.11.223.189:46057)
0621 08:01:02.130448 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.130452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.131098 (+ 646us) server_negotiation.cc:407] Connection header received
0621 08:01:02.131283 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.131286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.131334 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.131383 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.131881 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51046: BlockingRecv error: recv got EOF from 127.0.0.1:51046 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260621 08:01:02.137840 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.136115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.11.223.189:46057)
0621 08:01:02.136228 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.136232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.136841 (+ 609us) server_negotiation.cc:407] Connection header received
0621 08:01:02.137033 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.137037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.137088 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.137153 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.137718 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51060: BlockingRecv error: recv got EOF from 127.0.0.1:51060 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260621 08:01:02.146027 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.144115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.11.223.189:46057)
0621 08:01:02.144269 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.144273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.144973 (+ 700us) server_negotiation.cc:407] Connection header received
0621 08:01:02.145216 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.145219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.145280 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.145335 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.145901 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51072: BlockingRecv error: recv got EOF from 127.0.0.1:51072 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260621 08:01:02.159174 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.156692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.11.223.189:46057)
0621 08:01:02.156829 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.156834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.157828 (+ 994us) server_negotiation.cc:407] Connection header received
0621 08:01:02.158110 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.158114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.158187 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.158265 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.159035 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51076: BlockingRecv error: recv got EOF from 127.0.0.1:51076 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260621 08:01:02.179700 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.177910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.11.223.189:46057)
0621 08:01:02.178044 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.178049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.178656 (+ 607us) server_negotiation.cc:407] Connection header received
0621 08:01:02.178870 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.178874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.178929 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.178982 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.179575 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51092: BlockingRecv error: recv got EOF from 127.0.0.1:51092 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260621 08:01:02.215601 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.214473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.11.223.189:46057)
0621 08:01:02.214643 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.214647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.214816 (+ 169us) server_negotiation.cc:407] Connection header received
0621 08:01:02.214923 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.214926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.214974 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.215108 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.215499 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51096: BlockingRecv error: recv got EOF from 127.0.0.1:51096 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260621 08:01:02.283104 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.281949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.11.223.189:46057)
0621 08:01:02.282084 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.282088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.282235 (+ 147us) server_negotiation.cc:407] Connection header received
0621 08:01:02.282371 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.282373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.282423 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.282505 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.282973 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51104: BlockingRecv error: recv got EOF from 127.0.0.1:51104 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260621 08:01:02.414460 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.413390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.11.223.189:46057)
0621 08:01:02.413560 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.413564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.413619 (+ 55us) server_negotiation.cc:407] Connection header received
0621 08:01:02.413758 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.413760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.413806 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.413880 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.414331 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51106: BlockingRecv error: recv got EOF from 127.0.0.1:51106 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260621 08:01:02.673918 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0621 08:01:02.672725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.11.223.189:46057)
0621 08:01:02.672911 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:02.672918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:02.672955 (+ 37us) server_negotiation.cc:407] Connection header received
0621 08:01:02.673145 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:02.673149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:02.673202 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:02.673287 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0621 08:01:02.673792 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51074: BlockingRecv error: recv got EOF from 127.0.0.1:51074 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260621 08:01:02.963267 2569 catalog_manager.cc:1259] Loaded cluster ID: 9f92d49fec924324b92a8f733a5eb808
I20260621 08:01:02.963361 2569 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: loading cluster ID for follower catalog manager: success
I20260621 08:01:02.965240 2569 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: acquiring CA information for follower catalog manager: success
I20260621 08:01:02.966445 2569 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f7e4d624437485c8478ef3eb20a919c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 08:01:02.981782 2604 catalog_manager.cc:1259] Loaded cluster ID: 9f92d49fec924324b92a8f733a5eb808
I20260621 08:01:02.981873 2604 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: loading cluster ID for follower catalog manager: success
I20260621 08:01:02.983757 2604 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: acquiring CA information for follower catalog manager: success
I20260621 08:01:02.985131 2604 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b51ae552025f4af28b0c44f46e0adca4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260621 08:01:03.192003 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:03.187976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.11.223.189:46057)
0621 08:01:03.188222 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:03.188229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:03.188283 (+ 54us) server_negotiation.cc:407] Connection header received
0621 08:01:03.188580 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:03.188584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:03.188780 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:03.188914 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:03.189183 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.189716 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.190548 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.190783 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.190859 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:03.191143 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:03.191585 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:03.191702 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:03.191753 (+ 51us) server_negotiation.cc:299] Negotiation successful
0621 08:01:03.191803 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":66,"threads_started":1}
I20260621 08:01:03.198904 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:03.195211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.11.223.188:36367)
0621 08:01:03.195478 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:03.195481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:03.195495 (+ 14us) server_negotiation.cc:407] Connection header received
0621 08:01:03.195638 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:03.195641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:03.195760 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:03.195861 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:03.196050 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.196735 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.197517 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.197737 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.197816 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:03.198056 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:03.198494 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:03.198583 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:03.198635 (+ 52us) server_negotiation.cc:299] Negotiation successful
0621 08:01:03.198684 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":86,"threads_started":1}
I20260621 08:01:03.206575 2692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:03.201972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.11.223.190:35363)
0621 08:01:03.202400 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:03.202405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:03.202419 (+ 14us) server_negotiation.cc:407] Connection header received
0621 08:01:03.202524 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:03.202527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:03.202719 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:03.202844 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:03.203057 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.203681 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.204846 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.205198 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.205346 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:03.205696 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:03.206124 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:03.206254 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:03.206333 (+ 79us) server_negotiation.cc:299] Negotiation successful
0621 08:01:03.206411 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":81,"threads_started":1}
I20260621 08:01:03.206617 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:03.202025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.11.223.189:46057)
0621 08:01:03.202247 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:03.202251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:03.202639 (+ 388us) server_negotiation.cc:407] Connection header received
0621 08:01:03.202773 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:03.202778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:03.202844 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:03.202910 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:03.203261 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.204063 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.205185 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.205539 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.205654 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:03.206104 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:03.206193 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:03.206269 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:03.206446 (+ 177us) server_negotiation.cc:299] Negotiation successful
0621 08:01:03.206502 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260621 08:01:03.206822 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0621 08:01:03.202358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.11.223.188:36367)
0621 08:01:03.202458 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0621 08:01:03.202462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0621 08:01:03.202773 (+ 311us) server_negotiation.cc:407] Connection header received
0621 08:01:03.202829 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0621 08:01:03.202835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0621 08:01:03.202887 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0621 08:01:03.202964 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0621 08:01:03.203304 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.204070 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.205291 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0621 08:01:03.205636 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0621 08:01:03.205731 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0621 08:01:03.206013 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0621 08:01:03.206107 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0621 08:01:03.206193 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0621 08:01:03.206565 (+ 372us) server_negotiation.cc:299] Negotiation successful
0621 08:01:03.206639 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260621 08:01:03.209949 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 2330
I20260621 08:01:03.216122 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 2405
I20260621 08:01:03.222087 12158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKM8XKG/build/debug/bin/kudu with pid 2480
2026-06-21T08:01:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3094 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6193 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (275808 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST